-- Logs begin at Thu 2022-06-16 11:06:29 UTC, end at Thu 2022-06-16 12:14:47 UTC. -- Jun 16 11:36:20.377390 dv-cinder-846146-2-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jun 16 11:36:20.386193 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: *** Starting uWSGI 2.0.18-debian (64bit) on [Thu Jun 16 11:36:20 2022] *** Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: nodename: dv-cinder-846146-2-iscsi Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: machine: x86_64 Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: clock source: unix Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: pcre jit disabled Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: detected number of CPU cores: 4 Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: current working directory: / Jun 16 11:36:20.388211 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: detected binary path: /usr/bin/uwsgi-core Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: your processes number limit is 47680 Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: your memory page size is 4096 bytes Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: detected max file descriptor number: 2048 Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: lock engine: pthread robust mutexes Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: thunder lock: enabled Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jun 16 11:36:20.388746 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jun 16 11:36:20.433577 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: Python main interpreter initialized at 0x55fe288769c0 Jun 16 11:36:20.433577 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: python threads support enabled Jun 16 11:36:20.433577 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: your server socket listen backlog is limited to 100 connections Jun 16 11:36:20.433577 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: your mercy for graceful operations on workers is 90 seconds Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: mapped 403077 bytes (393 KB) for 2 cores Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: *** Operational MODE: preforking *** Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: *** uWSGI is running in multiple interpreter mode *** Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: spawned uWSGI master process (pid: 87079) Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: spawned uWSGI worker 1 (pid: 87091, cores: 1) Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: spawned uWSGI worker 2 (pid: 87092, cores: 1) Jun 16 11:36:20.460754 dv-cinder-846146-2-iscsi devstack@keystone.service[87079]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jun 16 11:36:20.434457 dv-cinder-846146-2-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jun 16 11:36:25.884639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 11:36:25.884639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: from cryptography.utils import int_from_bytes Jun 16 11:36:25.885408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 11:36:25.885408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: from cryptography.utils import int_from_bytes Jun 16 11:36:25.888888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 11:36:25.888888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: from cryptography.utils import int_from_bytes Jun 16 11:36:25.889344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jun 16 11:36:25.889344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: from cryptography.utils import int_from_bytes Jun 16 11:36:25.933778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 11:36:25.933966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 11:36:25.934157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] command line args: [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 11:36:25.934425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 11:36:25.934600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ================================================================================ {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 11:36:25.934853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] admin_token = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.935101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] config_dir = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.935289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.935471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] config_source = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.935651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] debug = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.935849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.936360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.936445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jun 16 11:36:25.936514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.936577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jun 16 11:36:25.936640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.936702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] command line args: [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jun 16 11:36:25.936891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.936976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jun 16 11:36:25.937115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ================================================================================ {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jun 16 11:36:25.937233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_config_append = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] admin_token = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] config_dir = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_dir = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_file = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.937914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] config_source = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] debug = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] insecure_debug = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.938930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_config_append = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.939956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_dir = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_file = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.940941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] publish_errors = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.941938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.942946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] standard_threads = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] notification_format = cadf {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.943952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] use_journal = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.40/identity {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] use_json = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] use_stderr = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944345 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] publish_errors = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] use_syslog = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944768 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.944987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.945154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.945242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.945391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.945468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.945627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] standard_threads = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.945709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.945861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] strict_password_check = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.946015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.946094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.946168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.946273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] use_eventlog = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.946348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.946451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] use_journal = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.946591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.946674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] use_json = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.946817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] use_stderr = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.946899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.946994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] use_syslog = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.947071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.947229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] watch_log_file = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jun 16 11:36:25.947306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.947487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.947564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.947727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.947810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.948984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.949968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.950998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.951992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.952916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.953869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.954819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.955987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.956909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.957989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.958998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.959993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.960897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.961056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.961131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.961289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.961372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.961811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.961896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.962874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.963818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.964949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.enabled = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.965995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.966955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.967909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.968046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.968125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.968287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.968373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.968537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.968785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.969026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.969265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.969504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.969737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.969944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.970987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.external = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.971912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.972009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.972231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.972469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.password = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.972705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] auth.token = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.972951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.973192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.973445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.973611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.973701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.973885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.973963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.external = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.mapped = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.974956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.password = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] auth.token = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.975979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] catalog.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.976957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.977126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.977202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.977372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.977452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.977630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.977712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.978399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.978478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.978645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.978730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.978890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.978966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.979966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.980958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.981930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.982904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.983900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.984855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.985993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.986965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.987924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.988977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.989949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.990906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.991851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.992988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.993074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.993228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.993304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.993529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.password = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.994985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.995986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.996937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.997959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.998931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:25.999898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.000870 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001268 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.001987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.002999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.003946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] policy.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.004949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] receipt.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.005920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] role.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] role.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.006855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.007849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] revoke.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] role.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.008992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] role.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] role.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] role.list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.009839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.010908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.011922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.012912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013870 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.013955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.014903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.015856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.016867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.017971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.provider = fernet {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.018928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.connection = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.019961 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] trust.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.020922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.021941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.connection = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.022895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.023974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87091) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 11:36:26.024048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.024288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.024537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.024816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.025067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.025308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.025559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.025798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.026045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.026280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.026525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.026765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jun 16 11:36:26.026937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=87092) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jun 16 11:36:26.139890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.140151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.140344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.143108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.143349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.143585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.148779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.152417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.177391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.177620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.178678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.178910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.231163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.231788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.279066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.279290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.280617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.280854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.344275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.344502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.344724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.344856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.344984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.345909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.368584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.368816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.368950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.369927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.370059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.370188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.370318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.370318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.380713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.381089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.393358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.393586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.393715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.393855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.393984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394110 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.394886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87091) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.395012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.395012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=87092) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jun 16 11:36:26.411921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.412134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 11:36:26.412266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.412428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 11:36:26.412563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.412706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 11:36:26.412848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.412980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jun 16 11:36:26.413108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.413317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.413450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.413588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.413717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.413845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.413973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.414103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.425184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.425458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.425727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.425859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.425987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.426114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.426256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.426519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.434043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.434637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.437414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.437693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.437972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.438101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.438257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.438387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.438523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.438653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.438779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.438906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.439975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.440104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.440231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.440392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.440658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.467002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.467817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.468632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.468907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.469446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.469730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.474109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.474391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.474666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.474965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.475244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.475523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.482670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.482909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.483494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.483775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.487355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.487638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.487938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.488218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.488346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.488488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.488626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.488768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.488899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.489030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.489160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.489296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.489425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.489555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.489821 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.490087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.507783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.508081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.508777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.509655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.513148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.514224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.516785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.517084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.517898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.518193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.522157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.523315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.526409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.527814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.530121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.530416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.530695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.530972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.531246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.531466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.531548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.531764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.532073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.532358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.532636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.532921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.550550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.550828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.551091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.551349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.552033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.552317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.552583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.552845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.560189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.560815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.561885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.562168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.562524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.562801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.565209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.565493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.565868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.566154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.567079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.567381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.567664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.567778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.568108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.568402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.581360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.581638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.581925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.582200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.582283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.582474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.582557 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.582741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.582825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.583018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.583101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.583380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.583655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.583954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.601723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.602697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.604998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.606002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.608296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.609326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.609872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.610921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.611638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.611946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.612228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.612716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.612998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.613279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.619466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.620584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.621102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.622193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.624336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.624620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.624896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.625165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.625449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.625736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.626291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.626573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.635142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.635423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.635696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.636353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.636650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.636930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.643552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87091) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.643829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.644131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.644404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.644678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.644761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=87092) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jun 16 11:36:26.644951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.645842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.646979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.647150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87091) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.647271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.647542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.647811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.648103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.648374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=87092) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jun 16 11:36:26.685148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fe288769c0 pid: 87091 (default app) Jun 16 11:36:26.686413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x55fe288769c0 pid: 87092 (default app) Jun 16 11:36:26.688589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8c36-3b9e-4a27-9e51-15776ff61998 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:26.688891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8c36-3b9e-4a27-9e51-15776ff61998 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:26.689172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201f8c36-3b9e-4a27-9e51-15776ff61998 None None] PATH_INFO: `/v3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:26.693348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1/1] 10.222.0.40 () {50 vars in 791 bytes} [Thu Jun 16 11:36:26 2022] GET /identity/v3/ => generated 255 bytes in 8 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:35.414202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:35.414767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:35.414767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:35.419090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1/2] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:35 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:36:35.424329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:35.424757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:35.424897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:35.500587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: INFO dbcounter [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] Registered counter for database keystone Jun 16 11:36:35.521477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG oslo_db.sqlalchemy.engines [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87091) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 11:36:35.527846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writer thread running {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 11:36:35.702563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG passlib.handlers.bcrypt [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87091) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 11:36:35.702992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG passlib.handlers.bcrypt [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87091) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 11:36:35.757441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:35.759066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:35.878781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 2/3] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:35 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 11:36:36.027818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:36.097130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: INFO dbcounter [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] Registered counter for database keystone Jun 16 11:36:36.116213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG oslo_db.sqlalchemy.engines [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=87092) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jun 16 11:36:36.122283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writer thread running {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jun 16 11:36:36.239634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37583d45-4ea3-4e6b-bfa6-be64a1060e6d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:36.244456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:36.245119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:36.245404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:36.245681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:36.253664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:36.304939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 11:36:36.305232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ed3275-7003-406c-aecb-3720eb4180b0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:36.315240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 2/4] 10.222.0.40 () {58 vars in 1324 bytes} [Thu Jun 16 11:36:36 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:36:36.321290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:36.338127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d6d4e6e-b96b-4c70-866b-8b23c0a46f80 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:36.342567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:36.343679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:36.344003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:36.344284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] PATH_INFO: `/v3/projects/c9c201c143164f7b8064c986ec08f397` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:36.345796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] RBAC: Authorizing `identity:get_project(project_id=c9c201c143164f7b8064c986ec08f397)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:36.396756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING keystone.common.rbac_enforcer.enforcer [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jun 16 11:36:36.397102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0ea84e-4bb7-4f8e-821f-153915c043a6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:36.401905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 3/5] 10.222.0.40 () {56 vars in 1207 bytes} [Thu Jun 16 11:36:36 2022] GET /identity/v3/projects/c9c201c143164f7b8064c986ec08f397 => generated 344 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:38.502697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a8354-0b58-444a-a7dd-eb4cdf65be2b None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:38.503227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a8354-0b58-444a-a7dd-eb4cdf65be2b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:38.503320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-766a8354-0b58-444a-a7dd-eb4cdf65be2b None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:38.507380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 3/6] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:38 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:38.513297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:38.513744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:38.513876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:38.532959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:38.534509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:38.542439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 4/7] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:38 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jun 16 11:36:38.563806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a8354-0b58-444a-a7dd-eb4cdf65be2b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:38.576757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-766a8354-0b58-444a-a7dd-eb4cdf65be2b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:38.580985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f1ba04-5917-42e7-a4e4-cc833e4a95de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:38.581629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1ba04-5917-42e7-a4e4-cc833e4a95de admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:38.581919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1ba04-5917-42e7-a4e4-cc833e4a95de admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:38.582201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f1ba04-5917-42e7-a4e4-cc833e4a95de admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:38.590730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1ba04-5917-42e7-a4e4-cc833e4a95de admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:38.592535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1f1ba04-5917-42e7-a4e4-cc833e4a95de admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:38.601232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 4/8] 10.222.0.40 () {58 vars in 1324 bytes} [Thu Jun 16 11:36:38 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 11:36:38.606530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:38.615463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de25a752-d0e5-498c-aeb8-fa3cd862917f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:38.619835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bb188a9-01a9-4d2d-84dd-77790ce9e2cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:38.620720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb188a9-01a9-4d2d-84dd-77790ce9e2cd admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:38.621007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb188a9-01a9-4d2d-84dd-77790ce9e2cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:38.621286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bb188a9-01a9-4d2d-84dd-77790ce9e2cd admin admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:38.622675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb188a9-01a9-4d2d-84dd-77790ce9e2cd admin admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:38.624537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bb188a9-01a9-4d2d-84dd-77790ce9e2cd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:38.626098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 5/9] 10.222.0.40 () {56 vars in 1198 bytes} [Thu Jun 16 11:36:38 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 20 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:40.785312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045879-486d-4106-abd7-f53092308809 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:40.785839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045879-486d-4106-abd7-f53092308809 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:40.785839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16045879-486d-4106-abd7-f53092308809 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:40.787206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 5/10] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:40.792345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:40.792764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:40.793051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:40.809501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:40.810684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:40.818530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450d2b-9d8a-4955-97b1-0d1784530eb7 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:40.818831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450d2b-9d8a-4955-97b1-0d1784530eb7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:40.819114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6450d2b-9d8a-4955-97b1-0d1784530eb7 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:40.820495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 6/11] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:36:40.825891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:40.826188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:40.826476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:40.865830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG passlib.handlers.bcrypt [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=87092) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jun 16 11:36:40.866263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG passlib.handlers.bcrypt [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=87092) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jun 16 11:36:40.891103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 6/12] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:36:40.914209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:40.914356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:40.915910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:40.936429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.014318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d838449-fc1e-40ea-8537-1c46242855d4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.014868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 7/13] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:36:41.036764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.047436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.048267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.048556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.048833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.060006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:41.071674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:41.072804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:36:41.073886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 7/14] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:36:40 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:36:41.078672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.137095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e913c87d-d7be-45ec-b9ef-7ae639f80cb6 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.139362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.158858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.173483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.174379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.174665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.174938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.176108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aabe6a8-dcb9-4490-b4b0-06e00a32accd None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.209830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.210549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.210841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.211122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.212224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:41.212721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.214666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.226236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 8/15] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:41.231655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.242552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.244586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.329512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a4fcf7-acc5-4652-b4e3-212c29b34cfc None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.339857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 8/16] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:36:41.345395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.347994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.362587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.363456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.363740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.364055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.365267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.366715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.369013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 9/17] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:41.373997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.389463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.442250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c3a6ead-e863-44b9-8a53-1f66db3baed5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.473279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d87df0-ab1f-4eb6-981f-8030ab5bf38f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.476974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.478265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.478549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.478826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.497268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.498748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.506605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c2892-a397-485a-8335-046c40e96105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.507403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c2892-a397-485a-8335-046c40e96105 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.507686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c2892-a397-485a-8335-046c40e96105 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.507991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323c2892-a397-485a-8335-046c40e96105 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.509165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c2892-a397-485a-8335-046c40e96105 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.510585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323c2892-a397-485a-8335-046c40e96105 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.512201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 10/18] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/domains/default => generated 199 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:41.517740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c2892-a397-485a-8335-046c40e96105 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:36:41.519005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:41.519986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:41.520970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:36:41.520970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:36:41.520970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:36:41.520970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:36:41.520970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 9/19] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:36:41.525585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:41.580373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.599118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:41.617610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323c2892-a397-485a-8335-046c40e96105 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.623404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16a9e045-7bd7-4037-b1c8-3f8535617bf9 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:41.651147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.651806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.652136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.652416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.653242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.domain.id=default, user.id=1361568ee63a477ea9c9be31d039c80d) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:41.653554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.654981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d76ba6e2-94ce-4db8-ac6b-d2b0153afefa None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.655545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:41.656226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:41.656513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:41.656792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:41.657513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:41.657823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:41.659192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e893a8df-73eb-4ce4-98e6-7b6e03427d06 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:41.661548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 11/20] 10.222.0.40 () {56 vars in 1322 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/role_assignments?user.id=1361568ee63a477ea9c9be31d039c80d&scope.domain.id=default&role.id=fda22db004a64c87b35510b321190ad4 => generated 233 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:41.679079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 10/21] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:36:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:44.084854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02062070-bd47-49c5-87ad-0f146c3dafc0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.085401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02062070-bd47-49c5-87ad-0f146c3dafc0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.085401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02062070-bd47-49c5-87ad-0f146c3dafc0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.087379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 12/22] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:44.093566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.094082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.094153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.116985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:44.121628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:44.208789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ebcf-2f74-4661-921d-e1a2c4feb0ad None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.209078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ebcf-2f74-4661-921d-e1a2c4feb0ad None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.209518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8927ebcf-2f74-4661-921d-e1a2c4feb0ad None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.211077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 13/23] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:36:44.216498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 11/24] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:44.217061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3aa1d-8bfe-4720-8fe7-2b555c537912 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.217511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3aa1d-8bfe-4720-8fe7-2b555c537912 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.217936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54b3aa1d-8bfe-4720-8fe7-2b555c537912 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.252490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-54b3aa1d-8bfe-4720-8fe7-2b555c537912 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:44.253904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-54b3aa1d-8bfe-4720-8fe7-2b555c537912 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:44.257259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:44.336128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:44.372443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 14/25] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:36:44.374768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f6c9-e126-49ac-ace1-6d89c8b30bb3 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.377047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f6c9-e126-49ac-ace1-6d89c8b30bb3 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.377498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab7f6c9-e126-49ac-ace1-6d89c8b30bb3 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.379130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 15/26] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:44 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:44.385027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.388623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.389050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.422278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:44.423758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:44.443582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b52b2783-90f2-4ecd-b372-5a4e7f85fc3e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:44.517842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:44.519560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.520190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.521954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.537495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:44.547260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:44.548320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:44.549241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:36:44.551389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 12/27] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:36:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:36:44.552413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:44.574254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 16/28] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:44.575082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:44.582031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:44.706784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-921017f4-9b7d-4e6d-8400-36e438b11b23 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:44.714726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:44.747041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:44.748110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.749555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.749984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.757501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf4ea5b-ab8e-407a-9748-02db5b055d55 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:44.760915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:44.762529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:36:44.790614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:44.791596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:44.792991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:36:44.792991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:36:44.792991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:36:44.792991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:36:44.795811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 13/29] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 244 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:36:44.797741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:44.843275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:44.845533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:44.845964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:44.846399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:44.847298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:44.849094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:44.853130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:44.869765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 17/30] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:36:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:44.872832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:44.966010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccab79f4-e602-4e88-9e4f-07bb52a8f6c5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.009266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfa5bd3-99c9-4595-9021-ad9e32903840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.009797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:45.010195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5bd3-99c9-4595-9021-ad9e32903840 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.010818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5bd3-99c9-4595-9021-ad9e32903840 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.011123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfa5bd3-99c9-4595-9021-ad9e32903840 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.013060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5bd3-99c9-4595-9021-ad9e32903840 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.014450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdfa5bd3-99c9-4595-9021-ad9e32903840 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.056745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d47a7709-0bce-4eb1-922b-f5e7e265f8d3 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.093194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.093932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.094147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.094422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.095155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:45.095463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.096875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.121035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 18/31] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:36:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 243 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:45.121035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:45.240348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f1dede-9701-4bd9-b8d6-e135f798a15f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.241358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 14/32] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 11:36:44 2022] POST /identity/v3/roles => generated 219 bytes in 445 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:36:45.243667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1d69-7be9-4fb8-8c62-3be46a4e80f9 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.244214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1d69-7be9-4fb8-8c62-3be46a4e80f9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.244710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312d1d69-7be9-4fb8-8c62-3be46a4e80f9 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.246451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 15/33] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:45.247376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d1d69-7be9-4fb8-8c62-3be46a4e80f9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:45.272595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:45.283093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:45.297377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb7d233-6525-452f-a585-c10149dbc9d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.298887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7d233-6525-452f-a585-c10149dbc9d7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.299376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7d233-6525-452f-a585-c10149dbc9d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.299852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb7d233-6525-452f-a585-c10149dbc9d7 None admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.302454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7d233-6525-452f-a585-c10149dbc9d7 None admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.308711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fb7d233-6525-452f-a585-c10149dbc9d7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.310973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 19/34] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:45 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:45.312569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.316871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.316871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.345201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:45.346744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:45.443467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312d1d69-7be9-4fb8-8c62-3be46a4e80f9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.487152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 20/35] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:45.487781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:45.511251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.512442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.512928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.513433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.514641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.516473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.623924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8b6f36-9dbc-424a-a7d3-477e2d6b4fd5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.641218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 16/36] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:36:45 2022] POST /identity/v3/projects => generated 301 bytes in 394 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:36:45.642501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:45.666187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:45.682087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2611bb39-daed-405f-9d40-82556088daa6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.683472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611bb39-daed-405f-9d40-82556088daa6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.683970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611bb39-daed-405f-9d40-82556088daa6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.684663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2611bb39-daed-405f-9d40-82556088daa6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.691548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611bb39-daed-405f-9d40-82556088daa6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.694440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2611bb39-daed-405f-9d40-82556088daa6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.698008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 21/37] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:45 2022] GET /identity/v3/domains/default => generated 199 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:45.699018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cccadd-fc7b-4aad-9e36-a9300eaacf6a None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.699385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cccadd-fc7b-4aad-9e36-a9300eaacf6a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.699762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4cccadd-fc7b-4aad-9e36-a9300eaacf6a None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.703946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 22/38] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:45.703946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fd96-1ac2-41b0-8c7c-b3f0d7f86f8b None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.703946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fd96-1ac2-41b0-8c7c-b3f0d7f86f8b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.703946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0665fd96-1ac2-41b0-8c7c-b3f0d7f86f8b None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.704531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 23/39] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:45.705504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665fd96-1ac2-41b0-8c7c-b3f0d7f86f8b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:45.750810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:45.785018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abab2f3d-b14b-4b79-9538-c820a37f6af8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.831801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0665fd96-1ac2-41b0-8c7c-b3f0d7f86f8b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:45.832862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b60963a-5f12-4ff2-9cb8-b63706a34810 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.833922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60963a-5f12-4ff2-9cb8-b63706a34810 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.833922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60963a-5f12-4ff2-9cb8-b63706a34810 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.834088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b60963a-5f12-4ff2-9cb8-b63706a34810 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.834803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60963a-5f12-4ff2-9cb8-b63706a34810 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.836253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b60963a-5f12-4ff2-9cb8-b63706a34810 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.868771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:45.869683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.869979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.870262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] PATH_INFO: `/v3/domains/default/users/1361568ee63a477ea9c9be31d039c80d/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:45.872318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=1361568ee63a477ea9c9be31d039c80d, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:45.873791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:45.938990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 17/40] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 11:36:45 2022] POST /identity/v3/roles => generated 225 bytes in 297 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:36:45.940300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36867668-bf1d-433c-bd6d-b21f09ed7efc None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:45.940600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36867668-bf1d-433c-bd6d-b21f09ed7efc None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:45.940887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36867668-bf1d-433c-bd6d-b21f09ed7efc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:46.071026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-36867668-bf1d-433c-bd6d-b21f09ed7efc None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:46.073016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-36867668-bf1d-433c-bd6d-b21f09ed7efc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:46.154995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 18/41] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:45 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 215 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:46.156132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:46.156423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:46.156700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:46.175999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:46.177179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:46.209931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 24/42] 10.222.0.40 () {58 vars in 1361 bytes} [Thu Jun 16 11:36:45 2022] PUT /identity/v3/domains/default/users/1361568ee63a477ea9c9be31d039c80d/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 505 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:36:46.211946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:46.246143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:46.259754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 19/43] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:46.271812 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:46.282172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:46.328056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65d7dd96-b97b-4e5e-abbe-f6354432b5a9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:46.363573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:46.364450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:46.364743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:46.365028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:46.366361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:46.369041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:46.374876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 25/44] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:46.380466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:46.404851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fec04c5-95c0-4e84-938b-48baadf1803b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:46.439589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aec0360b-bb46-4997-b9ff-1bb4ece3fb37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:46.440301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0360b-bb46-4997-b9ff-1bb4ece3fb37 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:46.440592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0360b-bb46-4997-b9ff-1bb4ece3fb37 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:46.440884 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aec0360b-bb46-4997-b9ff-1bb4ece3fb37 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:46.441646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0360b-bb46-4997-b9ff-1bb4ece3fb37 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:46.443034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aec0360b-bb46-4997-b9ff-1bb4ece3fb37 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:46.488865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29304b65-09ac-4be7-a8ef-71955f570108 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:46.509430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:46.519933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 20/45] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 11:36:46 2022] POST /identity/v3/roles => generated 223 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:36:46.525019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca1c668-e6f8-4dad-b057-e898314977f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:46.525723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c668-e6f8-4dad-b057-e898314977f8 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:46.526020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c668-e6f8-4dad-b057-e898314977f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:46.526333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca1c668-e6f8-4dad-b057-e898314977f8 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:46.527053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c668-e6f8-4dad-b057-e898314977f8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:46.528703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca1c668-e6f8-4dad-b057-e898314977f8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:46.640071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 26/46] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:36:46 2022] POST /identity/v3/projects => generated 312 bytes in 260 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:36:48.305470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e40fe2-55bf-4e73-8157-47211e95d2a9 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.305981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e40fe2-55bf-4e73-8157-47211e95d2a9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.306066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e40fe2-55bf-4e73-8157-47211e95d2a9 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.307410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 21/47] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:48.312704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.313118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.313274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.330340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:48.331858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:48.396836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 27/48] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:48.420173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e40fe2-55bf-4e73-8157-47211e95d2a9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:48.470787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:48.520269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e40fe2-55bf-4e73-8157-47211e95d2a9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:48.554045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:48.554846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.555166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.555450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.565695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:48.573895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:48.574688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:36:48.575446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 22/49] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:36:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:36:48.580870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:48.678120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8db42209-32dc-47ea-baa1-6c9bb8c05bbe None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:48.680462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:48.717643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:48.718319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.718605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.718885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.719651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:48.719992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:48.721389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37633800-5ddd-4692-8d13-3693a0d45aae None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:48.729597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 28/50] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:36:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:48.734969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:48.748820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf57f-06a1-47d8-aba2-c6cc467eff64 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.749120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf57f-06a1-47d8-aba2-c6cc467eff64 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.749411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71cbf57f-06a1-47d8-aba2-c6cc467eff64 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.750779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 29/51] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:36:48.755783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.756111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.756400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.773066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:48.774294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:48.834040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-399fe4ed-b279-483c-a809-a3b23bb346c0 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:48.839017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 30/52] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:48.844450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:48.861699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:48.866917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:48.868322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.868615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.868898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.870361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:48.871835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:48.873509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 23/53] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:48 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:48.878832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:48.953631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:48.960252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-befd98f1-9562-4d54-87a4-6b19eae2b41b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:48.981546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7f497c-29e5-4d81-94b6-b1142e8e7cbc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:48.994377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:48.995184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:48.995473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:48.995754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:48.996969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:48.998429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:49.000085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 31/54] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:49.005841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:49.015051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:49.015855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:49.016180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:49.016463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:49.017663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:49.019125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:49.020776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 24/55] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:48 2022] GET /identity/v3/domains/default => generated 199 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:49.026332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:49.071724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:49.102793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfc536ab-55e0-4653-8129-be2d06f8b903 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:49.126349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-034eb393-a603-410e-a8b8-fffab50e77ce None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:49.136770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f13341c-c4a6-48f0-9645-156915e6e1a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:49.137436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13341c-c4a6-48f0-9645-156915e6e1a8 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:49.137725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13341c-c4a6-48f0-9645-156915e6e1a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:49.138006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f13341c-c4a6-48f0-9645-156915e6e1a8 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:49.138712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13341c-c4a6-48f0-9645-156915e6e1a8 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:49.140233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f13341c-c4a6-48f0-9645-156915e6e1a8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:49.159330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:49.160019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:49.160312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:49.160594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:49.161418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.domain.id=default, user.id=1361568ee63a477ea9c9be31d039c80d) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:49.161734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:49.163151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-889abfad-dd57-442b-a5c6-d13808710f65 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:49.172094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 25/56] 10.222.0.40 () {56 vars in 1322 bytes} [Thu Jun 16 11:36:49 2022] GET /identity/v3/role_assignments?user.id=1361568ee63a477ea9c9be31d039c80d&scope.domain.id=default&role.id=fda22db004a64c87b35510b321190ad4 => generated 530 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:49.342373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 32/57] 10.222.0.40 () {60 vars in 1140 bytes} [Thu Jun 16 11:36:49 2022] POST /identity/v3/projects => generated 298 bytes in 337 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:36:51.426696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d765f-29c8-45b8-af0a-1d4abd54b021 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:51.427210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d765f-29c8-45b8-af0a-1d4abd54b021 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:51.427293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f73d765f-29c8-45b8-af0a-1d4abd54b021 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:51.428699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 26/58] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:51.433671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:51.434051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:51.434236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:51.450975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:51.452198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:51.461024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:51.518037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 33/59] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:51.543651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d765f-29c8-45b8-af0a-1d4abd54b021 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:51.644183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f73d765f-29c8-45b8-af0a-1d4abd54b021 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:51.653981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:51.676222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0505ac0-1114-409b-8efa-c3163e7bde63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:51.677065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505ac0-1114-409b-8efa-c3163e7bde63 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:51.677305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505ac0-1114-409b-8efa-c3163e7bde63 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:51.677583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0505ac0-1114-409b-8efa-c3163e7bde63 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:51.678745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0505ac0-1114-409b-8efa-c3163e7bde63 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:51.680743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0505ac0-1114-409b-8efa-c3163e7bde63 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:51.682396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 27/60] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:51 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:51.687943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:51.782548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9669dac7-0b3e-4480-91d7-691f34fe2693 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:51.814797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3eb9e4b-372c-4b05-9c5e-bdcb218be5ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:51.815459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb9e4b-372c-4b05-9c5e-bdcb218be5ec None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:51.815744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb9e4b-372c-4b05-9c5e-bdcb218be5ec None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:51.816053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb9e4b-372c-4b05-9c5e-bdcb218be5ec None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:51.816767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb9e4b-372c-4b05-9c5e-bdcb218be5ec None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:51.818220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3eb9e4b-372c-4b05-9c5e-bdcb218be5ec None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:51.924636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:51.929129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 34/61] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:36:51 2022] POST /identity/v3/users => generated 275 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:36:55.084040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a5919-7434-40fe-8805-d0c88601baa1 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.084892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a5919-7434-40fe-8805-d0c88601baa1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.085323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c5a5919-7434-40fe-8805-d0c88601baa1 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.087163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 28/62] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:55.092831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.093668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.094219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.115467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:55.120977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:55.168171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4cce-f45a-466d-87a2-9fcfc5cb113e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.168693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4cce-f45a-466d-87a2-9fcfc5cb113e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.169155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a04a4cce-f45a-466d-87a2-9fcfc5cb113e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.170678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 29/63] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:55 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:55.175747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.176221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.176648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.205016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:55.206263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:55.210419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 35/64] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:55.248447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:55.307992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 30/65] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:36:55.331638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:55.440975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f87cb93-6b95-421c-8520-c2429df26224 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:55.496516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2be4efc-7bdd-43cc-a76d-115297b06a8a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:55.498448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:55.512820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:55.524357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.524357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.524357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:55.529952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:55.538233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:55.539122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:36:55.540741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:36:55.542849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 36/66] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:36:55 2022] GET /identity/v3/roles/member => generated 84 bytes in 295 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:36:55.556868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:55.574988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d382221b-2bf1-46c1-97e5-8265ffdd5e1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:55.575815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382221b-2bf1-46c1-97e5-8265ffdd5e1d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.576131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382221b-2bf1-46c1-97e5-8265ffdd5e1d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.576417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d382221b-2bf1-46c1-97e5-8265ffdd5e1d None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.592579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d382221b-2bf1-46c1-97e5-8265ffdd5e1d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d382221b-2bf1-46c1-97e5-8265ffdd5e1d None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:55.603837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:55.604696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:36:55.605423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:36:55.608283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 31/67] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:36:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:36:55.614540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba81b-abca-49aa-8377-83270b32d54b None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.614540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba81b-abca-49aa-8377-83270b32d54b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.614540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-749ba81b-abca-49aa-8377-83270b32d54b None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.615944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 32/68] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:55 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:55.616897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba81b-abca-49aa-8377-83270b32d54b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:55.698108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-295f60e8-c087-47e0-8eee-1a56caecb4d1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:55.734580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-749ba81b-abca-49aa-8377-83270b32d54b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:55.739956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:55.740644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.740939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.741225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.741993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:55.742343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:55.743783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02c61746-5c56-44cb-9745-c2e8fa9750a8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:55.753341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 37/69] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:36:55 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:55.754675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.754972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.755259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.773283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:55.775211 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:55.776130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:55.776835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.777132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.777470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.778262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:55.779344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:55.780077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a69049a6-3bc1-4f4c-9272-51d440a02a10 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:55.800074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 33/70] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:36:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:55.801470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ed95c-46f9-4f45-b0ed-64457262007a None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.801769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ed95c-46f9-4f45-b0ed-64457262007a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.802062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da7ed95c-46f9-4f45-b0ed-64457262007a None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.807302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 34/71] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:55.807302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ed95c-46f9-4f45-b0ed-64457262007a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:55.831587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:55.838110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:55.852247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 38/72] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:55.853076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:55.910418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da7ed95c-46f9-4f45-b0ed-64457262007a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:55.944228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be496757-3283-448e-905b-ff112f241234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:55.945104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496757-3283-448e-905b-ff112f241234 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.945914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496757-3283-448e-905b-ff112f241234 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.946199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be496757-3283-448e-905b-ff112f241234 None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.955302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-061f35c3-5690-49b8-babb-f336f43985a7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:55.960572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496757-3283-448e-905b-ff112f241234 None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:55.962844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be496757-3283-448e-905b-ff112f241234 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:55.964539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 35/73] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:55 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:55.965700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.965995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.966278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.982571 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:55.983784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:55.993396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f09bbcbd-f4e8-4bfd-a5e0-8194262d912e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:55.994278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bbcbd-f4e8-4bfd-a5e0-8194262d912e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:55.994564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bbcbd-f4e8-4bfd-a5e0-8194262d912e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:55.994840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f09bbcbd-f4e8-4bfd-a5e0-8194262d912e None admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:55.996084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bbcbd-f4e8-4bfd-a5e0-8194262d912e None admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:55.997520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f09bbcbd-f4e8-4bfd-a5e0-8194262d912e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:55.999134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 39/74] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:55 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:56.000317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82248-af06-488c-b3aa-a012fff902d8 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.000607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82248-af06-488c-b3aa-a012fff902d8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.000888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e82248-af06-488c-b3aa-a012fff902d8 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.002255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 40/75] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:36:55 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:36:56.002806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82248-af06-488c-b3aa-a012fff902d8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.055282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 36/76] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:55 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:56.055461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.056769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.075567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.107740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e82248-af06-488c-b3aa-a012fff902d8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.141744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.142554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.142841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.143116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.153142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:56.157847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d41fd00-ca85-41ba-bee3-bf86feeefd04 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:56.161231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:56.162167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 41/77] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:36:56.167850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.191749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38295d03-44ab-4796-986c-e9e157765ca1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.192436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38295d03-44ab-4796-986c-e9e157765ca1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.192728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38295d03-44ab-4796-986c-e9e157765ca1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.193012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38295d03-44ab-4796-986c-e9e157765ca1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.258960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38295d03-44ab-4796-986c-e9e157765ca1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.260399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38295d03-44ab-4796-986c-e9e157765ca1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.262664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cc83696-a258-4d1a-81e6-feb8b0a3b479 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.272655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.277480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.295228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.295913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.296213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.296503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.355972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 37/78] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:36:56.357308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.357601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.357882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.367939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.369462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.373692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:36:56.374879 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:36:56.439502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 38/79] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:36:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:36:56.440336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.473752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 42/80] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:36:56.474604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.485934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.490815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.540011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9abb7087-af81-450b-bd19-038b8b360ca8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.571488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc1e4e9d-dc9f-4db2-ab77-91b46a1afe73 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.573271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.574067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.574356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.574629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.575786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.577265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.578872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 39/81] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:56.580760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.605443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.606117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.606411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.606700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.607535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:56.607766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.609257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.617052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 43/82] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:56.618445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.677392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f865398-ecbb-406c-afcf-85e5a7bfb6f6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.690100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.694564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.711566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.712719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.713016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.713298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.714507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.715990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.717732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 40/83] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:56.718502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.718978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2106bdf8-fae8-41c6-93a8-9990e37de024 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.753961 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.754759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.755043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.755320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.765520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:36:56.773257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:36:56.774104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:36:56.775031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 44/84] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/roles/member => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:36:56.776352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.825427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfe9c759-6764-4c33-b2a2-87f67e6ecef4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.858056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.858923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.859216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.859503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.860745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.862202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.863930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 41/85] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:56.864697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:56.881290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26589d2c-d4f2-455a-b331-c8f74acacb0c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.909483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.914261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.914932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.915217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.915497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.916251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.917698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:56.924628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:56.963037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c4c633e-8eee-4b09-99dc-0c9ba1e8a62d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:56.994323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:56.995186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:56.995474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:56.995749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:56.996973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:56.998416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.000054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 42/86] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:56 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:57.000804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.037963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 45/87] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:36:56 2022] POST /identity/v3/users => generated 289 bytes in 263 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:36:57.038893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.112511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b395629a-34bd-441a-add3-2b422838d85f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:57.135932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:57.152646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:57.153893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:57.154760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:57.155393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:57.156660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=05e24c3335b843adafa09ce1a2c81793) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:57.157139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:57.157584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f517c2b3-ad7d-4cac-8b72-29f354e25b9b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:57.159538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.180140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 43/88] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/role_assignments?user.id=05e24c3335b843adafa09ce1a2c81793&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:57.180140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.231585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:57.232404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:57.232695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:57.233152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:57.233931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:57.234853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:57.236636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.249436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 46/89] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:57.250318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.323335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:57.341935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1064367-fb0b-416f-a1b0-bf5a5fe90833 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:57.407368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdbf0baa-1bb9-4f4b-83dd-87579c3bd29d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:57.408644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:57.409524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:57.409987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:57.410461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:57.411504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=1361568ee63a477ea9c9be31d039c80d) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:57.411825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:57.413312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.419921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 44/90] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/role_assignments?user.id=1361568ee63a477ea9c9be31d039c80d&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 251 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:57.420822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.451331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:57.452527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:57.453108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:57.453605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:57.476423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:57.529899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9da51dd2-f570-4808-83b1-0017ae3efe12 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:57.567507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:57.571896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:57.571896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:57.571896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:57.571896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:57.572471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.574192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 45/91] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:57.580214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.584348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:57.586969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.697725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c641ce9-be43-4367-8547-b4854dfb7c0f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:57.716294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:57.736398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:57.746075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:57.746993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:57.747429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:57.747855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:57.842575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:57.846649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 47/92] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 596 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:36:57.847508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:57.850959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.971611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 46/93] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 392 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:36:57.977731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:57.997979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:58.044634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3ef3f6d-d789-4982-bbeb-6fb14e8e9803 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:58.092839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6b20cd-f71e-49e5-9418-a8f9c15a5c03 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:58.122239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:58.123198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:58.123452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:58.123745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:58.128885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:58.130365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:58.132162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 48/94] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:58.134012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:58.135092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:58.135527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:58.136051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] PATH_INFO: `/v3/projects/21b911efc85f4fcea0d4d799dd93bbae` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:58.137830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] RBAC: Authorizing `identity:get_project(project_id=21b911efc85f4fcea0d4d799dd93bbae)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:58.138474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:58.139716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:58.141712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 47/95] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:36:57 2022] GET /identity/v3/projects/21b911efc85f4fcea0d4d799dd93bbae => generated 312 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:36:58.153099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:36:58.190529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:58.268650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94cd5f1f-2be9-4294-9954-97717bf83e97 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:58.308199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:58.309103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:58.309574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:58.310320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:58.311393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=21b911efc85f4fcea0d4d799dd93bbae, user.id=05e24c3335b843adafa09ce1a2c81793) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:58.311983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:58.313679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cada5e15-9cb1-4f8a-a7fa-4114aaf547e2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:58.319677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 48/96] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:36:58 2022] GET /identity/v3/role_assignments?user.id=05e24c3335b843adafa09ce1a2c81793&scope.project.id=21b911efc85f4fcea0d4d799dd93bbae&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 259 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:36:58.320235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:36:58.387200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e24f73d-7cc9-4e00-af2a-94bb68a3a14e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:36:58.438528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:36:58.439218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:36:58.439509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:36:58.439788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:36:58.440670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] RBAC: Adding query filter params (role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=05e24c3335b843adafa09ce1a2c81793) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:36:58.440989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:36:58.442462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b076a18-3902-4045-9647-80ddadb8e1c4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:36:58.452779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 49/97] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:36:58 2022] GET /identity/v3/role_assignments?user.id=05e24c3335b843adafa09ce1a2c81793&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c4622473a5824ed7876be10e3da526ca => generated 259 bytes in 314 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:00.066031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43915df3-1a25-480d-8a2a-270fb91175d7 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.066999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43915df3-1a25-480d-8a2a-270fb91175d7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.067491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43915df3-1a25-480d-8a2a-270fb91175d7 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.069109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 49/98] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:00.084514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89003c-d7a8-489d-b16e-cb783df8f042 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.084990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89003c-d7a8-489d-b16e-cb783df8f042 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.085062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c89003c-d7a8-489d-b16e-cb783df8f042 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.112176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8c89003c-d7a8-489d-b16e-cb783df8f042 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:00.113395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8c89003c-d7a8-489d-b16e-cb783df8f042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:00.208839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 50/99] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:00.209015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8831cc-9ebe-441d-b9f5-7a06ebe91f07 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.209501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8831cc-9ebe-441d-b9f5-7a06ebe91f07 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.210038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a8831cc-9ebe-441d-b9f5-7a06ebe91f07 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.212432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 50/100] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:00.217884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd165f92-416a-4240-82b2-112e39a5085a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.218182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd165f92-416a-4240-82b2-112e39a5085a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.218466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd165f92-416a-4240-82b2-112e39a5085a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.239520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-cd165f92-416a-4240-82b2-112e39a5085a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:00.240746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-cd165f92-416a-4240-82b2-112e39a5085a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:00.251291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:00.266170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8831cc-9ebe-441d-b9f5-7a06ebe91f07 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:00.443650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a8831cc-9ebe-441d-b9f5-7a06ebe91f07 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:00.459815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 51/101] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:00.483246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ea8a3-db87-4c85-b5e9-2760448e596d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.483556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ea8a3-db87-4c85-b5e9-2760448e596d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.483830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d6ea8a3-db87-4c85-b5e9-2760448e596d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.485507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 52/102] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:00.493528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.493863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.494104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.511778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:00.512657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.512953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.513241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.528377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:00.536264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:00.537079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:00.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:00.538437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:00.538512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 51/103] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:00 2022] GET /identity/v3/roles/reader => generated 84 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:00.538573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:00.538643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:00.552177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:00.637587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 53/104] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 145 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:00.639176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:00.643675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f1e719f-eef9-4241-be0c-81ddcbc58d8f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:00.681118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e484699-0520-4335-bb85-0543562f492b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:00.682188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e484699-0520-4335-bb85-0543562f492b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.682599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e484699-0520-4335-bb85-0543562f492b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.683037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e484699-0520-4335-bb85-0543562f492b None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.699085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e484699-0520-4335-bb85-0543562f492b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-8e484699-0520-4335-bb85-0543562f492b None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:00.706938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:00.707786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 52/105] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:00.712371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b5b6-aee4-417f-b5a6-b41143554e1f None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.714953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b5b6-aee4-417f-b5a6-b41143554e1f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.714953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a563b5b6-aee4-417f-b5a6-b41143554e1f None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.714953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 53/106] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:00 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:00.714953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563b5b6-aee4-417f-b5a6-b41143554e1f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:00.746536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:00.784269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:00.786703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d0a013-35cf-4d15-ac98-b0391f411e86 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:00.816196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a563b5b6-aee4-417f-b5a6-b41143554e1f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:00.833537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:00.834223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.834519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.835061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.835861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] RBAC: Adding query filter params (name=reader) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:00.836236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:00.838283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15f6d5f8-3db4-4725-97b8-876054c8605d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:00.850578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 54/107] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:00 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:00.852193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.852503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.853319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.857306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:00.858102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:00.858395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:00.858678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:00.870459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:00.874608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:00.876209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:00.878388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:00.879253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:00.880459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 54/108] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:00 2022] GET /identity/v3/roles/admin => generated 83 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:00.881298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:00.973984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 55/109] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:00.975584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:00.991401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dd80b07-3dae-48b4-9a7f-d8ee8ba458a9 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.029194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.042936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.044180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.044767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.044934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.050367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.050367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:01.050367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.050367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.061887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 55/110] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.061887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.095416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fb68e0f-469c-49df-b811-ec2dcb33fe7d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.135018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.136094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.136539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.136820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] PATH_INFO: `/v3/users/07b2dc53ce78426590f517a8a1e98ffd` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.155749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] RBAC: Authorizing `identity:get_user(user_id=07b2dc53ce78426590f517a8a1e98ffd)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.157295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.160607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 56/111] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:00 2022] GET /identity/v3/users/07b2dc53ce78426590f517a8a1e98ffd => generated 289 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.161519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.185764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d8a924-81d4-4c89-8cb0-c181376fd91a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.222482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.223520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.224094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.224575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.225544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:01.226053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.228197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.237124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 56/112] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.238325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.287635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1865be08-6ef6-4ad2-a0b4-c1015ca47328 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.308837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.312354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.324346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.325413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.325903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.326404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.327820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.329514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.331357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 57/113] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.332344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.371085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a87ceb1-a1b1-429d-8965-b5fa8aa9f89e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.414829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.415725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.416050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.416335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.417986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.419455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.421287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 57/114] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.422066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.452380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8643cd52-27c4-43ed-b89d-b5f5dff251ca None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.506108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.506760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.507056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.507339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.553395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fe4a53-6bea-4510-9e9f-dfd56825346e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.571228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.572818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.580988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.582479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.586676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.587966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.588134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.588474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.589661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.591102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.592751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 58/115] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.593490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.679599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 58/116] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:01.680479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.694531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efb45237-0369-4bf5-a9a7-561085638cec None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.729412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9976d176-f16d-4f37-8ab4-d7ebe7ae44d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.730072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d176-f16d-4f37-8ab4-d7ebe7ae44d6 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.730361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d176-f16d-4f37-8ab4-d7ebe7ae44d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.730644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9976d176-f16d-4f37-8ab4-d7ebe7ae44d6 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.731348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976d176-f16d-4f37-8ab4-d7ebe7ae44d6 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.732830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9976d176-f16d-4f37-8ab4-d7ebe7ae44d6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.771073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.780814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c08cb64b-9a62-4fa6-93f9-d9a3df334d70 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.812475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.813344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.813819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.814104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.815218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 59/117] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:37:01 2022] POST /identity/v3/projects => generated 302 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:01.815326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.816552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924851e-d7cd-46e4-844f-c5eb9e939cce None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.816823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.816916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924851e-d7cd-46e4-844f-c5eb9e939cce None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.817115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0924851e-d7cd-46e4-844f-c5eb9e939cce None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.818523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 60/118] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:01.818626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 59/119] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.819668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.824579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943f3a0-9cc6-4559-8790-1e8d890623fb None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.824869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943f3a0-9cc6-4559-8790-1e8d890623fb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.825150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9943f3a0-9cc6-4559-8790-1e8d890623fb None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.826788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 61/120] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:01 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:01.827359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943f3a0-9cc6-4559-8790-1e8d890623fb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:01.879316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:01.940572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a8cdb69-8040-477a-9d77-280d36a078c6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.946837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9943f3a0-9cc6-4559-8790-1e8d890623fb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:01.982082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07afccf8-269b-4291-8361-ef9397d4b001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.983449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afccf8-269b-4291-8361-ef9397d4b001 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.983449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afccf8-269b-4291-8361-ef9397d4b001 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.983689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07afccf8-269b-4291-8361-ef9397d4b001 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.985147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afccf8-269b-4291-8361-ef9397d4b001 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.986347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07afccf8-269b-4291-8361-ef9397d4b001 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.988104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 60/121] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.989277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584c7bf-7f1b-4edd-b79a-a3f4e96668ed None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.989571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584c7bf-7f1b-4edd-b79a-a3f4e96668ed None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.989851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1584c7bf-7f1b-4edd-b79a-a3f4e96668ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.992474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:01.993715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:01.994165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:01.994449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:01.995753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:01.997708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:01.999110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 62/122] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:01 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:01.999960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.011012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1584c7bf-7f1b-4edd-b79a-a3f4e96668ed None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:02.011411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1584c7bf-7f1b-4edd-b79a-a3f4e96668ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:02.037259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:02.087724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 61/123] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:01 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:02.089061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:02.089351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:02.089636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:02.118631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:02.119839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:02.121437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68b5cff4-61e4-442d-b30b-bd5a16281c16 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:02.132185 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:02.159940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:02.162120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:02.162120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:02.162120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735/users/05e24c3335b843adafa09ce1a2c81793/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:02.163757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] RBAC: Authorizing `identity:create_grant(project_id=85416ef9b7144e4889bdac4457d53735, user_id=05e24c3335b843adafa09ce1a2c81793, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:02.165607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:02.189484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 62/124] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:02.190321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.267466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 63/125] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:01 2022] PUT /identity/v3/projects/85416ef9b7144e4889bdac4457d53735/users/05e24c3335b843adafa09ce1a2c81793/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:02.269210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.291625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:02.309356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7046c7d5-17d8-4ed7-b292-68d4d1e01847 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:02.352067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:02.353180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:02.353566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:02.353965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735/users/1361568ee63a477ea9c9be31d039c80d/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:02.359099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] RBAC: Authorizing `identity:create_grant(project_id=85416ef9b7144e4889bdac4457d53735, user_id=1361568ee63a477ea9c9be31d039c80d, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:02.359099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:02.408746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fcfa435-f833-4f9f-8141-7ff14136ab8e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:02.437485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 63/126] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:02 2022] PUT /identity/v3/projects/85416ef9b7144e4889bdac4457d53735/users/1361568ee63a477ea9c9be31d039c80d/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:02.440448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.464604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:02.465865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:02.466363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:02.466838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:02.468176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] RBAC: Adding query filter params (role.id=30c1267a61504942b5b1382cc1d47901, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=07b2dc53ce78426590f517a8a1e98ffd) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:02.468705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:02.470715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:02.479410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 64/127] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:02 2022] GET /identity/v3/role_assignments?user.id=07b2dc53ce78426590f517a8a1e98ffd&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=30c1267a61504942b5b1382cc1d47901 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:02.480918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.549228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:02.643599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd5f851-35b6-43cc-8d9c-854b31c9a94a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:02.705467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05829b68-2c6d-493b-a8b1-f2602e173c35 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:02.726890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:02.728123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:02.728590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:02.729058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:02.752175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:02.762190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:02.763078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:02.763852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:02.770015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 64/128] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 332 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:02.777463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.778644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:02.780027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:02.782387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:02.783566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:02.794762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:02.809654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:02.817168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:02.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:02.818779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 65/129] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:02 2022] GET /identity/v3/roles/member => generated 84 bytes in 338 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:02.824517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:02.965237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:02.971656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5f78484-f29c-419f-80f2-930177d741bf None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:02.994748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34b724a-a51d-4d0a-97a1-717af05f4338 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.019907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.019907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.019907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.019907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.020297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:03.020375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.021750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.031492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 66/130] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:02 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:03.036830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:03.076663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.077711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.078203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.078691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.079723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:03.080266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.081907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.103600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 65/131] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:02 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:03.120953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:03.175313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b069fef-45c2-4404-b6d4-349a8d30eec6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.264527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:03.268899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.270444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.270937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.271410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.272883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.275239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.277158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 67/132] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:03 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:03.284823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:03.301289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806809f4-2b1f-46c8-99a0-49aa1b701809 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.327915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:03.340318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.341757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.341757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.342188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.343663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.346122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.347901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 66/133] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:03 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:03.354806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:03.462623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75017e51-f784-4c25-91d6-748e37c7c4b6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.473248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e42a852e-10a3-4b65-af84-8e0235cbad68 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.512663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.513803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.514341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.514629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.515861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.518094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.519799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 67/134] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:03 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:03.526076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:03.536591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.537119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:03.537495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.537795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.538087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] PATH_INFO: `/v3/projects/21b911efc85f4fcea0d4d799dd93bbae` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.539464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] RBAC: Authorizing `identity:get_project(project_id=21b911efc85f4fcea0d4d799dd93bbae)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.540976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.542971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 68/135] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:03 2022] GET /identity/v3/projects/21b911efc85f4fcea0d4d799dd93bbae => generated 312 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:03.548558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:03.704083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b502c6f8-6661-4554-b280-55c722250f16 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.771602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:03.787309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3fb6fa-8f7b-4ddd-80ec-dbbc5106bca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.788276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fb6fa-8f7b-4ddd-80ec-dbbc5106bca0 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.788581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fb6fa-8f7b-4ddd-80ec-dbbc5106bca0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.789407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3fb6fa-8f7b-4ddd-80ec-dbbc5106bca0 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735/users/05e24c3335b843adafa09ce1a2c81793/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.793578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadd8fe-40f7-4aaa-b1fd-35632ce8074b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:03.795696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fb6fa-8f7b-4ddd-80ec-dbbc5106bca0 None admin] RBAC: Authorizing `identity:create_grant(project_id=85416ef9b7144e4889bdac4457d53735, user_id=05e24c3335b843adafa09ce1a2c81793, role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.797209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3fb6fa-8f7b-4ddd-80ec-dbbc5106bca0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.854212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557eb70d-8e59-4da8-ba69-f907abd53bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:03.855124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eb70d-8e59-4da8-ba69-f907abd53bfa None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:03.855416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eb70d-8e59-4da8-ba69-f907abd53bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:03.855702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557eb70d-8e59-4da8-ba69-f907abd53bfa None admin] PATH_INFO: `/v3/projects/21b911efc85f4fcea0d4d799dd93bbae/users/05e24c3335b843adafa09ce1a2c81793/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:03.861325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eb70d-8e59-4da8-ba69-f907abd53bfa None admin] RBAC: Authorizing `identity:create_grant(project_id=21b911efc85f4fcea0d4d799dd93bbae, user_id=05e24c3335b843adafa09ce1a2c81793, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:03.862792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-557eb70d-8e59-4da8-ba69-f907abd53bfa None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:03.901026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 68/136] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:03 2022] PUT /identity/v3/projects/85416ef9b7144e4889bdac4457d53735/users/05e24c3335b843adafa09ce1a2c81793/roles/c4622473a5824ed7876be10e3da526ca => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:03.929614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 69/137] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:03 2022] PUT /identity/v3/projects/21b911efc85f4fcea0d4d799dd93bbae/users/05e24c3335b843adafa09ce1a2c81793/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:37:04.435847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945c7dc-8a3f-453a-a3f6-e4b0e841ffe0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:04.436388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945c7dc-8a3f-453a-a3f6-e4b0e841ffe0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:04.436467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8945c7dc-8a3f-453a-a3f6-e4b0e841ffe0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:04.438001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 69/138] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:04.443258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:04.443663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:04.443837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:04.461001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:04.462619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:04.549553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 70/139] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:04.593323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945c7dc-8a3f-453a-a3f6-e4b0e841ffe0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:04.692919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:04.742872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8945c7dc-8a3f-453a-a3f6-e4b0e841ffe0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:04.787038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a7b05-2723-499c-b56a-d3485ba088e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:04.787857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7b05-2723-499c-b56a-d3485ba088e1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:04.788167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7b05-2723-499c-b56a-d3485ba088e1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:04.788452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a7b05-2723-499c-b56a-d3485ba088e1 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:04.790189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7b05-2723-499c-b56a-d3485ba088e1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:04.791647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a7b05-2723-499c-b56a-d3485ba088e1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:04.793326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 70/140] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:04 2022] GET /identity/v3/domains/default => generated 199 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:04.798903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:04.906889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:04.935723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3df4411-e249-4a8b-8267-775ccb0fcf3a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:04.970705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac8e8ba-9e47-4bdf-b8f7-81143c404fcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:04.971395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e8ba-9e47-4bdf-b8f7-81143c404fcf None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:04.976832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e8ba-9e47-4bdf-b8f7-81143c404fcf None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:04.976832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac8e8ba-9e47-4bdf-b8f7-81143c404fcf None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:04.976832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e8ba-9e47-4bdf-b8f7-81143c404fcf None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:04.976832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac8e8ba-9e47-4bdf-b8f7-81143c404fcf None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:05.084530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 71/141] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:04 2022] POST /identity/v3/users => generated 283 bytes in 286 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:05.485761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6999b0-da16-496f-8934-92d9fc681332 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.486291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6999b0-da16-496f-8934-92d9fc681332 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.486365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6999b0-da16-496f-8934-92d9fc681332 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.487715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 71/142] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:05.493884 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7947a-c3cf-4c18-9dcc-29c819f9914e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.494278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7947a-c3cf-4c18-9dcc-29c819f9914e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.494439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf7947a-c3cf-4c18-9dcc-29c819f9914e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.511769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8bf7947a-c3cf-4c18-9dcc-29c819f9914e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:05.512985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8bf7947a-c3cf-4c18-9dcc-29c819f9914e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:05.587064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 72/143] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:05.633595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6999b0-da16-496f-8934-92d9fc681332 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:05.698126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f63e6-2f58-4120-99ba-13669a7fd358 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.698433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f63e6-2f58-4120-99ba-13669a7fd358 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.698723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-649f63e6-2f58-4120-99ba-13669a7fd358 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.702329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 73/144] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:05 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:05.707306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.707601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.707905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.727859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:05.729575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:05.761214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6999b0-da16-496f-8934-92d9fc681332 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:05.794724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:05.796790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:05.801618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:05.802741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.803299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.803756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.804592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 74/145] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:05.816917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:05.824269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:05.825090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:05.825819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:05.828644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 72/146] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:05.829188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:05.842267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:05.938762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f823aea7-fb94-461f-a822-0670b06167e7 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:05.949205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e99a98c-bd31-476a-9c8b-e7550d61607e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:05.977081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3c56522-813a-41a5-b1b7-ae0052184cf5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:05.977889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56522-813a-41a5-b1b7-ae0052184cf5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.978171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56522-813a-41a5-b1b7-ae0052184cf5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.978668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3c56522-813a-41a5-b1b7-ae0052184cf5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.986330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:05.987285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:05.987894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:05.988385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:05.989338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:05.989778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3c56522-813a-41a5-b1b7-ae0052184cf5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:05.990053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:05.991625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-c3c56522-813a-41a5-b1b7-ae0052184cf5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:05.998210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:05.999053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:06.000475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 75/147] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:05 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:06.001107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bcb38-ef75-48ec-9803-a62c792af4bf None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.007973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bcb38-ef75-48ec-9803-a62c792af4bf None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.007973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bcb38-ef75-48ec-9803-a62c792af4bf None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.008367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 73/148] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:06.008791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 76/149] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:05 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:06.009686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.014530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.014530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.014722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.034906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:06.036206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:06.074733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:06.110485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 77/150] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:06.111378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.114536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:06.122827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78c6ce6-642c-4115-b13e-f02d1679f914 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.164652 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.165697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.166204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.166691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.167647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:06.168214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:06.169883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:06.183694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 74/151] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:06.184896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.245710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-781ba376-8b43-40cd-8b8a-d52778912343 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.281964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.282767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.283050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.283325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.284548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:06.286003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:06.287633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 78/152] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:06.294444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.314035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adee5e4f-46ca-4aa2-a4a7-9e71db1d040c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.340267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:06.358242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:06.358632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.359997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.360662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.361473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.383918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:06.392459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:06.393520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:06.394352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 75/153] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:06.394977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.427417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bdb41be-1473-4763-90f1-7f6c5ff4eff7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.495212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.496141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.496429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.496698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] PATH_INFO: `/v3/users/1361568ee63a477ea9c9be31d039c80d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.498891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] RBAC: Authorizing `identity:get_user(user_id=1361568ee63a477ea9c9be31d039c80d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:06.500396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:06.502109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 79/154] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/users/1361568ee63a477ea9c9be31d039c80d => generated 247 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:06.503170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.579204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a81618b-177c-4442-9b12-5d48eb6c64dc None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.619387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.620079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.620367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.620647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.650234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1501b782-7976-4c09-b286-81d9633137b3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.686579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:06.690056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:06.713207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc757154-c031-4978-9197-166a317cf74e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.714040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757154-c031-4978-9197-166a317cf74e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.714331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757154-c031-4978-9197-166a317cf74e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.717733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc757154-c031-4978-9197-166a317cf74e None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:06.717733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc757154-c031-4978-9197-166a317cf74e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:06.717733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc757154-c031-4978-9197-166a317cf74e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:06.717733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc757154-c031-4978-9197-166a317cf74e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:06.726051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 80/155] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:06.727427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc757154-c031-4978-9197-166a317cf74e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.730976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:06.732866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:06.860604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc757154-c031-4978-9197-166a317cf74e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:06.871273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 76/156] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:06.872177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:06.925107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:06.925770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:06.926059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:06.926339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.021942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:07.027530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:07.046434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.048278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.104244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9c9da5-a41c-4704-bb23-f0a8d92189a1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.172453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 81/157] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 446 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:07.173849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:07.198019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.199204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.199728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.200235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] PATH_INFO: `/v3/users/07b2dc53ce78426590f517a8a1e98ffd` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.204636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] RBAC: Authorizing `identity:get_user(user_id=07b2dc53ce78426590f517a8a1e98ffd)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.206374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.208345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 77/158] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:06 2022] GET /identity/v3/users/07b2dc53ce78426590f517a8a1e98ffd => generated 289 bytes in 337 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:07.209329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:07.275275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:07.295099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a649377f-67fb-4e6b-b162-91d4fd6b67ad None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.337961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.339410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.339860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.340303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.341673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.344093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.346219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 82/159] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:07 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:07.352367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:07.470744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff7206a-d834-4e87-b80c-046cd5a59aa3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.478052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51d30cc-f7cc-4a0e-806f-249d87685fa2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.512061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.513917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.514469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.515022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.516528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.518441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.520494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 83/160] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:07 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:07.521548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:07.546518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:07.576611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25a840d7-4764-466c-9bde-a8bc7a50f398 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.577637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a840d7-4764-466c-9bde-a8bc7a50f398 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.577979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a840d7-4764-466c-9bde-a8bc7a50f398 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.578262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25a840d7-4764-466c-9bde-a8bc7a50f398 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.579528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a840d7-4764-466c-9bde-a8bc7a50f398 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.581022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25a840d7-4764-466c-9bde-a8bc7a50f398 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.582971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 78/161] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:07 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:07.584089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb42260-4588-4658-af6e-810f8b81d0d9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.584386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb42260-4588-4658-af6e-810f8b81d0d9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.584726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb42260-4588-4658-af6e-810f8b81d0d9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.585973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 79/162] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:07.593019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb42260-4588-4658-af6e-810f8b81d0d9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:07.626111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:07.648918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eb3d7a6-789e-46ed-8b4e-3f3f7890868a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.699154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.700155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.700785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.701308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.702420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=05e24c3335b843adafa09ce1a2c81793) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:07.702940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.704625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.713043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 84/163] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:07 2022] GET /identity/v3/role_assignments?user.id=05e24c3335b843adafa09ce1a2c81793&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 608 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:07.719582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:07.829168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:07.849961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e5e6716-8f37-4c73-8d81-fa584a52b7f4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.903315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.904429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.905010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.905673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.907114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=1361568ee63a477ea9c9be31d039c80d) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:07.907650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:07.909346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88280799-e92c-467f-b7e3-626163484be4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:07.917546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 85/164] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:07 2022] GET /identity/v3/role_assignments?user.id=1361568ee63a477ea9c9be31d039c80d&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:07.918924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb42260-4588-4658-af6e-810f8b81d0d9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:07.919401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f96fc-43f2-4667-ae7b-0f97d16673ee None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:07.920011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f96fc-43f2-4667-ae7b-0f97d16673ee None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:07.920655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-754f96fc-43f2-4667-ae7b-0f97d16673ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:07.956422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-754f96fc-43f2-4667-ae7b-0f97d16673ee None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:07.959196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-754f96fc-43f2-4667-ae7b-0f97d16673ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:07.997651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:07.999837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.003407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.003963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735/users/07b2dc53ce78426590f517a8a1e98ffd/roles/30c1267a61504942b5b1382cc1d47901` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.006341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] RBAC: Authorizing `identity:create_grant(project_id=85416ef9b7144e4889bdac4457d53735, user_id=07b2dc53ce78426590f517a8a1e98ffd, role_id=30c1267a61504942b5b1382cc1d47901)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:08.009905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:08.074582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 86/165] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:08.076578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100d8f6-b12a-4878-85ff-9f6fe44d4798 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.077108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100d8f6-b12a-4878-85ff-9f6fe44d4798 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.077592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2100d8f6-b12a-4878-85ff-9f6fe44d4798 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.079232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 87/166] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:08.085077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.085701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.086210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.099188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 80/167] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:07 2022] PUT /identity/v3/projects/85416ef9b7144e4889bdac4457d53735/users/07b2dc53ce78426590f517a8a1e98ffd/roles/30c1267a61504942b5b1382cc1d47901 => generated 0 bytes in 513 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:08.117852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:08.119553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:08.141600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:08.217861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 88/168] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:08.256107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:08.268168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:08.309599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:08.347528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a0e442-7a40-435f-93ba-cba2dbd2d95d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:08.406285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:08.407156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.407444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.407724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.426684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:08.437750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086a2e73-d29f-4af1-9e7c-c60c150dc4cd None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:08.439173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:08.440164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:08.440939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 81/169] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:08 2022] GET /identity/v3/roles/member => generated 84 bytes in 299 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:08.447810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:08.476060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:08.477511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.477799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.478076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.491047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:08.500415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:08.501306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:08.502075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 89/170] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:08 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 246 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:08.509896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:08.584352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:08.621568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff6a416-4dfb-4cab-bb94-a7e8a1a8438b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:08.680628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:08.681369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.681642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.681917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.682677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:08.682985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:08.684398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:08.702467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 90/171] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:08.707469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-212fbe46-753b-403f-9eb1-da71497837ec None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:08.708547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:08.762716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:08.787422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:08.788172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:08.788976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:08.789273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:08.790034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:08.790343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:08.791731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:08.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 82/172] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:08 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 374 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:08.827992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:08.945781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b7e50e-53bb-4761-9f21-2e39ea728fed None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:09.004928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15abf197-865f-4da5-8c2f-11548b63261b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:09.016318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:09.032762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:09.033797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.034123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.034400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.035846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:09.037430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:09.044176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 91/173] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:08 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 336 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:09.045640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:09.051461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:09.052375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.052658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.052932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] PATH_INFO: `/v3/users/05e24c3335b843adafa09ce1a2c81793` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.054776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] RBAC: Authorizing `identity:get_user(user_id=05e24c3335b843adafa09ce1a2c81793)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:09.056269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:09.058045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 83/174] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:08 2022] GET /identity/v3/users/05e24c3335b843adafa09ce1a2c81793 => generated 275 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:09.064118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:09.151351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:09.191316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c6f4f34-2fef-448c-8f6a-fda0a84ca164 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:09.233095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:09.233983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.234276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.234601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.318316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:09.320574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:09.385836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93cd5790-d4cb-4501-8284-09ce05b901c3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:09.410380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:09.471959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 84/175] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 408 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:09.477571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:09.568700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f8ce6b-e1b4-4266-b22d-662dedb81636 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:09.569292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8ce6b-e1b4-4266-b22d-662dedb81636 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.569581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8ce6b-e1b4-4266-b22d-662dedb81636 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.570394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f8ce6b-e1b4-4266-b22d-662dedb81636 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.668322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:09.669869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8ce6b-e1b4-4266-b22d-662dedb81636 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:09.672161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f8ce6b-e1b4-4266-b22d-662dedb81636 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:09.730234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5272d597-d201-4d17-b827-fcdb50b0c8fe None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:09.771171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824ca702-4363-43df-a4eb-ccf871b0155b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:09.772115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ca702-4363-43df-a4eb-ccf871b0155b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.772403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ca702-4363-43df-a4eb-ccf871b0155b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.772681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824ca702-4363-43df-a4eb-ccf871b0155b None admin] PATH_INFO: `/v3/projects/21b911efc85f4fcea0d4d799dd93bbae` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.773985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ca702-4363-43df-a4eb-ccf871b0155b None admin] RBAC: Authorizing `identity:get_project(project_id=21b911efc85f4fcea0d4d799dd93bbae)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:09.775459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824ca702-4363-43df-a4eb-ccf871b0155b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:09.783117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 85/176] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:09 2022] GET /identity/v3/projects/21b911efc85f4fcea0d4d799dd93bbae => generated 312 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:09.784504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3d4fe-9c23-4181-b680-1f92bdc8f4a0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.784800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3d4fe-9c23-4181-b680-1f92bdc8f4a0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.785087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ad3d4fe-9c23-4181-b680-1f92bdc8f4a0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.786510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 86/177] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:09.787770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955a805-148a-41bc-96b0-e77b898e3dc0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.788452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955a805-148a-41bc-96b0-e77b898e3dc0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.788743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8955a805-148a-41bc-96b0-e77b898e3dc0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.789990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 87/178] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:09.791101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955a805-148a-41bc-96b0-e77b898e3dc0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:09.803268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 92/179] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 758 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:09.805329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce6531-7e70-4b7c-8e12-918728bc28ed None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.805622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce6531-7e70-4b7c-8e12-918728bc28ed None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.805905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69ce6531-7e70-4b7c-8e12-918728bc28ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.823017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-69ce6531-7e70-4b7c-8e12-918728bc28ed None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:09.824163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-69ce6531-7e70-4b7c-8e12-918728bc28ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:09.837640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:09.894381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 93/180] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:09.896081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.896376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.896830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.918593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8955a805-148a-41bc-96b0-e77b898e3dc0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:09.922057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:09.923500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:09.948388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:09.956730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:09.957397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.957690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.962259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.962259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=21b911efc85f4fcea0d4d799dd93bbae, user.id=05e24c3335b843adafa09ce1a2c81793) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:09.962259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:09.962259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a6383e-87e7-4396-9753-4f51283b2425 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:09.967182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 88/181] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:09 2022] GET /identity/v3/role_assignments?user.id=05e24c3335b843adafa09ce1a2c81793&scope.project.id=21b911efc85f4fcea0d4d799dd93bbae&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 608 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:09.969784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35725934-6eca-4a03-99f5-189373e9eee8 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:09.970371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35725934-6eca-4a03-99f5-189373e9eee8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:09.970883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35725934-6eca-4a03-99f5-189373e9eee8 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:09.977894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 89/182] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:09.977894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35725934-6eca-4a03-99f5-189373e9eee8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.004984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 94/183] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:10.012812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.118201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.130127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35725934-6eca-4a03-99f5-189373e9eee8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.138948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3804a186-6c64-4651-b4fe-d4377b45a60d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.178849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28febf96-202f-43d2-9bde-6c865c755e80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.180539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28febf96-202f-43d2-9bde-6c865c755e80 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.180959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28febf96-202f-43d2-9bde-6c865c755e80 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.181397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28febf96-202f-43d2-9bde-6c865c755e80 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.182109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.182909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28febf96-202f-43d2-9bde-6c865c755e80 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:10.183067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.183364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.183647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.184650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28febf96-202f-43d2-9bde-6c865c755e80 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:10.187205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 90/184] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:09 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:10.188311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.188596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.188872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.203104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:10.206081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:10.207266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:10.211064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:10.212498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:10.213294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 95/185] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/roles/admin => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:10.213945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.237377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.299601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 91/186] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:10.299601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.331079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-905bdb7b-81aa-457b-917f-ba6bc4b93905 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.366058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.367497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.367789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.368092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.369340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:10.370758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:10.372613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 96/187] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:10.373193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.403860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.412559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0400af61-4738-4bb1-a539-15c81d689cb9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.451468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.452164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.452454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.452737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.453586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] RBAC: Adding query filter params (role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=05e24c3335b843adafa09ce1a2c81793) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:10.453908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:10.455806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:10.471906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 92/188] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/role_assignments?user.id=05e24c3335b843adafa09ce1a2c81793&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c4622473a5824ed7876be10e3da526ca => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:10.471906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.476917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.495341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6946b155-7808-4ac1-8c91-ce5b598174b5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.551346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.552084 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.552362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.552645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.553424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:10.553740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:10.555254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:10.564783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 97/189] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:10.565747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.607685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55473700-4d0e-4ceb-b30e-0d2283a42a65 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.648584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.649763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.649763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.649928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.664587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:10.674607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:10.675523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:10.676442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 93/190] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:10.677089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.703160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.704719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968a7c1-6729-45db-bc59-fc9861d8219b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.752907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59cedf09-d781-472a-b013-a4ece57df826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.755725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cedf09-d781-472a-b013-a4ece57df826 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.756301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cedf09-d781-472a-b013-a4ece57df826 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.756877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59cedf09-d781-472a-b013-a4ece57df826 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.758812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cedf09-d781-472a-b013-a4ece57df826 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:10.761315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59cedf09-d781-472a-b013-a4ece57df826 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:10.804874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64593c4f-7d93-4439-bbf6-33bed7781e79 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.838980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:10.839851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.840169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.840454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] PATH_INFO: `/v3/users/9a71fc60dc954122aa77370a9de40bef` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.853389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] RBAC: Authorizing `identity:get_user(user_id=9a71fc60dc954122aa77370a9de40bef)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:10.854846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:10.856518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 94/191] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/users/9a71fc60dc954122aa77370a9de40bef => generated 283 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:10.857276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.872660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 98/192] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:10 2022] POST /identity/v3/users => generated 297 bytes in 301 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:10.872660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa05253-2f2b-4217-be93-66234f6d7707 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:10.872660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa05253-2f2b-4217-be93-66234f6d7707 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:10.872660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa05253-2f2b-4217-be93-66234f6d7707 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:10.872660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 99/193] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:10.872660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa05253-2f2b-4217-be93-66234f6d7707 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:10.882700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.953094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:10.979565 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d752f92c-f38c-4a97-b590-b364089fdba0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:10.985640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa05253-2f2b-4217-be93-66234f6d7707 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.018952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.020019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.020494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.020772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.021546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:11.021859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.024201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a078659-30be-4dd9-8656-d01e350785b0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.026378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.026921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.027213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.027483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.035806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 95/194] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:11.037766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.038237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.038727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.056867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:11.058080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:11.117446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.119072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.123212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.146213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 96/195] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:11.147031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.230024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.259807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 100/196] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:11.260683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.280570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ef1a67-ce2a-4803-9cdf-65757d852b59 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.317129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.318014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.318298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.318576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] PATH_INFO: `/v3/users/9a71fc60dc954122aa77370a9de40bef` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.320403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] RBAC: Authorizing `identity:get_user(user_id=9a71fc60dc954122aa77370a9de40bef)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.321967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.324178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 97/197] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/users/9a71fc60dc954122aa77370a9de40bef => generated 283 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:11.325032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.390409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93872df3-3284-4e5e-8035-4f00f05d53ea None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.393222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.435901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.437637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.438619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.439192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.441151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fa93e-24cd-4475-8def-e2ad91ebdfc2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.453989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:11.464970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:11.466023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:11.466993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:11.469278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 101/198] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/roles/reader => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:11.472648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.490134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.491243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.492210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.492554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.494338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.496288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.498635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 98/199] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:11.504947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.509571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.592360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73fe0de-afb2-453a-ad96-61dccf2a5007 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.629000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.629716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.629969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.630252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.636093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba55335-87be-4863-8761-e87a4a33c570 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.662883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.677905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.678912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.679971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.680418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.681529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:11.682098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.689161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.698582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 99/200] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:11.700750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.704793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.706352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.771545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.812303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b254b8c-f932-4b80-a476-cb7f74f97aa3 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.813209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 102/201] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:11.814075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.850733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:11.852312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:11.852803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:11.853553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:11.854590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=9a71fc60dc954122aa77370a9de40bef) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:11.855140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:11.856829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:11.863131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 100/202] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/role_assignments?user.id=9a71fc60dc954122aa77370a9de40bef&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:11.864177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:11.893205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:11.981936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e51890-f2cd-4cdc-b7af-a8852a67e1a7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:11.994217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-323d7ca8-1ab0-4e21-a140-edaea08812cf None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:12.012385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:12.021883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:12.023759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:12.024319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:12.024763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:12.026281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:12.027971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:12.030158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 101/203] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:12.036330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:12.081525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:12.082428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:12.082714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:12.083050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] PATH_INFO: `/v3/users/07b2dc53ce78426590f517a8a1e98ffd` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:12.088613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] RBAC: Authorizing `identity:get_user(user_id=07b2dc53ce78426590f517a8a1e98ffd)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:12.090091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:12.092077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 103/204] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:11 2022] GET /identity/v3/users/07b2dc53ce78426590f517a8a1e98ffd => generated 289 bytes in 278 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:12.102066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:12.240114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92364e-380e-4539-a768-2fba558bcb46 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:12.283683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:12.284807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:12.285293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:12.285769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:12.286918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] RBAC: Adding query filter params (role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=9a71fc60dc954122aa77370a9de40bef) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:12.287552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:12.289266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:12.295890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16936ba-fc2c-469f-b5a9-5178aa8844c1 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:12.305768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 102/205] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:12 2022] GET /identity/v3/role_assignments?user.id=9a71fc60dc954122aa77370a9de40bef&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c4622473a5824ed7876be10e3da526ca => generated 259 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:12.326676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:12.331641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:12.332519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:12.340044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:12.340044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:12.409683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:12.411196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:12.525678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 104/206] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:12.529056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:12.600152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:12.650331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c783e37d-7cd8-496b-9e26-f39fbda98484 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:12.701792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811d4198-12eb-41c3-89de-f04629c33be9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:12.702737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4198-12eb-41c3-89de-f04629c33be9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:12.703027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4198-12eb-41c3-89de-f04629c33be9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:12.703309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811d4198-12eb-41c3-89de-f04629c33be9 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:12.708629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4198-12eb-41c3-89de-f04629c33be9 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:12.710098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-811d4198-12eb-41c3-89de-f04629c33be9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:12.712163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 103/207] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:12 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:12.718537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:12.761012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:12.830427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-664d6dae-8cc1-49d6-81d3-af6ab7d75019 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:12.867292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:12.867895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:12.868251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:12.868537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:12.869385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] RBAC: Adding query filter params (role.id=30c1267a61504942b5b1382cc1d47901, scope.project.id=85416ef9b7144e4889bdac4457d53735, user.id=07b2dc53ce78426590f517a8a1e98ffd) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:12.869838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:12.871305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb99e31a-0110-4d0c-9959-e222f4bdbe8a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:12.880036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 105/208] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:12 2022] GET /identity/v3/role_assignments?user.id=07b2dc53ce78426590f517a8a1e98ffd&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=30c1267a61504942b5b1382cc1d47901 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:13.298070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826d76-fd08-4a2a-bdcb-95b420fadf8c None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.299084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826d76-fd08-4a2a-bdcb-95b420fadf8c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.299594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00826d76-fd08-4a2a-bdcb-95b420fadf8c None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.301522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 104/209] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:13 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:13.309062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154db98-413d-46e4-a24d-f65aac3bafd2 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.309357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154db98-413d-46e4-a24d-f65aac3bafd2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.309640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e154db98-413d-46e4-a24d-f65aac3bafd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.332942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e154db98-413d-46e4-a24d-f65aac3bafd2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:13.334218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e154db98-413d-46e4-a24d-f65aac3bafd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:13.419733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 106/210] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:13.442431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00826d76-fd08-4a2a-bdcb-95b420fadf8c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:13.446084 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960b45c-5b72-408a-8681-d6772b762542 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.446387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960b45c-5b72-408a-8681-d6772b762542 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.446674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b960b45c-5b72-408a-8681-d6772b762542 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.448639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 107/211] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:13.453337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.453634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.453920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.472077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:13.473363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:13.497344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:13.556277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00826d76-fd08-4a2a-bdcb-95b420fadf8c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:13.557377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 108/212] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:13.581270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:13.586267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:13.595144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:13.596327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.596773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.602500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.608940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:13.617096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:13.617947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:13.618715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:13.621429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 105/213] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:13 2022] GET /identity/v3/roles/member => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:13.627157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:13.686125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c675c098-4f54-458a-8680-0ef9818ea904 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:13.723481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:13.724330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.724653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.725167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.726822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:13.728406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:13.731976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-672bb249-9590-4985-9514-ef8a64a103e9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:13.735323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 109/214] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:13 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:13.736196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:13.767976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:13.769010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.769491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.769986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.770138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:13.770951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:13.771361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:13.773435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:13.783462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 106/215] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:13 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:13.792352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:13.840922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:13.849987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b985b1-818c-4779-86db-fa5cf0049a71 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:13.887765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33f62eb0-1d5a-4989-a8ac-17e107a5582d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:13.888942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f62eb0-1d5a-4989-a8ac-17e107a5582d None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.889395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f62eb0-1d5a-4989-a8ac-17e107a5582d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.889846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33f62eb0-1d5a-4989-a8ac-17e107a5582d None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.890701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f62eb0-1d5a-4989-a8ac-17e107a5582d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:13.892515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33f62eb0-1d5a-4989-a8ac-17e107a5582d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:13.895144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06af9b13-d469-40a5-a9bc-d624f3b924f9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:13.930523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:13.931510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:13.931880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:13.932284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] PATH_INFO: `/v3/users/24f6f9cbeccc48c5acfcbe4611fd3917` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:13.946830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] RBAC: Authorizing `identity:get_user(user_id=24f6f9cbeccc48c5acfcbe4611fd3917)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:13.948352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:13.950328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 107/216] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:13 2022] GET /identity/v3/users/24f6f9cbeccc48c5acfcbe4611fd3917 => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:13.955450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:14.000928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 110/217] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:13 2022] POST /identity/v3/users => generated 297 bytes in 259 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:37:14.069394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ab6241-00ca-40ee-b098-cab71f37b260 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:14.080766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:14.109304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:14.109304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.109544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.109788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.184372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:14.185938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:14.305728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 108/218] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:13 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:14.310765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a701c3e-1c51-42b9-afa4-eaffa6413274 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.311057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a701c3e-1c51-42b9-afa4-eaffa6413274 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.311224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:14.311339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a701c3e-1c51-42b9-afa4-eaffa6413274 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.312723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 111/219] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:14.317676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.317968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.318242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.321922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:14.337710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:14.338907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:14.423553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe8da02-ab30-4cd1-833c-cb961a5b53e0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:14.426505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 112/220] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:14.458183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:14.468208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:14.469505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.469959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.470486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.471001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:14.472837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:14.474508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:14.477233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 109/221] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:14 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:14.483212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:14.577222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:14.582979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-144ee230-4bb0-4375-9f56-80305a018391 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:14.607246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7b18af-e9ff-47dd-ab07-88ef9ed65a3c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:14.642070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:14.642873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.643152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.643423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.650105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755839-a190-411e-8630-a5249f464a1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:14.650773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755839-a190-411e-8630-a5249f464a1d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.651054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755839-a190-411e-8630-a5249f464a1d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.651329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df755839-a190-411e-8630-a5249f464a1d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.652198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755839-a190-411e-8630-a5249f464a1d None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=24f6f9cbeccc48c5acfcbe4611fd3917) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:14.652515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755839-a190-411e-8630-a5249f464a1d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:14.653973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df755839-a190-411e-8630-a5249f464a1d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:14.666593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:14.670458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 110/222] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:14 2022] GET /identity/v3/role_assignments?user.id=24f6f9cbeccc48c5acfcbe4611fd3917&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 259 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:14.676760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:14.677635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:14.679809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 113/223] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:14 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:14.686280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755839-a190-411e-8630-a5249f464a1d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:14.859068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df755839-a190-411e-8630-a5249f464a1d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:14.913306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:14.914841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.915320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.915791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.916801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:14.917312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:14.918912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec58fae0-4bb3-4f42-b02c-07d83a3f170b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:14.932661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 111/224] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:14 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 248 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:14.937845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:14.957364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5ca6-f3a0-4d6e-b18e-26794ab5e329 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.957972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5ca6-f3a0-4d6e-b18e-26794ab5e329 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.958468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fce5ca6-f3a0-4d6e-b18e-26794ab5e329 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.960686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 112/225] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:14.968104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:14.968645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:14.969122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:14.980200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:14.996822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:14.998047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:15.040555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:15.101847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d106fd6-9d0a-4e24-acc0-d08a3019e020 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:15.130125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 113/226] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:15.158573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:15.159482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:15.159761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:15.160162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] PATH_INFO: `/v3/users/9a71fc60dc954122aa77370a9de40bef` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:15.164626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] RBAC: Authorizing `identity:get_user(user_id=9a71fc60dc954122aa77370a9de40bef)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:15.166203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:15.168192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 114/227] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:14 2022] GET /identity/v3/users/9a71fc60dc954122aa77370a9de40bef => generated 283 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:15.173743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:15.186363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:15.324682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2410b0b-1944-449c-ba84-06d134234149 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:15.396691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:15.404180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:15.406344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:15.416052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:15.416052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:15.416052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:15.416052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:15.416052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 114/228] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:15 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:15.418365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7119c69-2e09-4d6c-ac75-2c3e84fe2ea6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:15.422326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:15.487610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:15.488513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:15.488805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:15.489085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:15.508194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:15.512823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:15.528620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:15.529500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:15.530270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:15.538025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 115/229] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:15 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 352 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:15.538953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:15.619661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d18b1702-b667-4ee2-9945-6f1a3de68e6e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:15.693083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:15.693999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:15.694287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:15.695128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/9a71fc60dc954122aa77370a9de40bef/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:15.699310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] RBAC: Authorizing `identity:create_grant(project_id=90ef71bb81154cd8856ce17bcf6eb121, user_id=9a71fc60dc954122aa77370a9de40bef, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:15.700806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:15.728046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49eaa20e-9b6f-4d85-9b30-fa3a51d6c3e2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:15.767226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:15.767226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:15.767226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:15.767226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:15.767577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:15.767577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:15.767577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:15.775744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 116/230] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:15 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:15.781299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:15.798291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 115/231] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:15 2022] PUT /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/9a71fc60dc954122aa77370a9de40bef/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:15.858136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:15.904163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aefba250-e99f-4609-927c-0db017d1f444 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:15.941993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:15.942900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:15.943190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:15.943543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] PATH_INFO: `/v3/users/9a71fc60dc954122aa77370a9de40bef` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:15.945440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] RBAC: Authorizing `identity:get_user(user_id=9a71fc60dc954122aa77370a9de40bef)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:15.946907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:15.948890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 117/232] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:15 2022] GET /identity/v3/users/9a71fc60dc954122aa77370a9de40bef => generated 283 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:15.954479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:16.032594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:16.082820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8cde16-184c-4629-b4fc-46de37ef9beb None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:16.119606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f95df02d-1a4b-444a-acd6-6326a7287886 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:16.120519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95df02d-1a4b-444a-acd6-6326a7287886 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:16.120807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95df02d-1a4b-444a-acd6-6326a7287886 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:16.121087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f95df02d-1a4b-444a-acd6-6326a7287886 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:16.122348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95df02d-1a4b-444a-acd6-6326a7287886 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:16.123806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f95df02d-1a4b-444a-acd6-6326a7287886 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:16.125777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 116/233] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:15 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:16.131211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:16.244755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec54755d-a922-42cf-a8e9-12fca14769c8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:16.281356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:16.293583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8193919c-3233-44de-ba65-2a6a3ed26260 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:16.294477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193919c-3233-44de-ba65-2a6a3ed26260 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:16.294774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193919c-3233-44de-ba65-2a6a3ed26260 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:16.295054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8193919c-3233-44de-ba65-2a6a3ed26260 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/9a71fc60dc954122aa77370a9de40bef/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:16.297417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193919c-3233-44de-ba65-2a6a3ed26260 None admin] RBAC: Authorizing `identity:create_grant(project_id=90ef71bb81154cd8856ce17bcf6eb121, user_id=9a71fc60dc954122aa77370a9de40bef, role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:16.298875 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8193919c-3233-44de-ba65-2a6a3ed26260 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:16.404312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 118/234] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:16 2022] PUT /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/9a71fc60dc954122aa77370a9de40bef/roles/c4622473a5824ed7876be10e3da526ca => generated 0 bytes in 274 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:16.746359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64146-5a9f-4ca7-bc5c-078d3844db71 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:16.746848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64146-5a9f-4ca7-bc5c-078d3844db71 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:16.746920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a64146-5a9f-4ca7-bc5c-078d3844db71 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:16.748353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 117/235] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:16.753859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838267-9a55-4377-8002-1a5e636e4e6a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:16.754385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838267-9a55-4377-8002-1a5e636e4e6a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:16.754886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b838267-9a55-4377-8002-1a5e636e4e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:16.773412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7b838267-9a55-4377-8002-1a5e636e4e6a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:16.774878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7b838267-9a55-4377-8002-1a5e636e4e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:16.848604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 119/236] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:16.872324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64146-5a9f-4ca7-bc5c-078d3844db71 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:16.928395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f3d10-013d-42f2-9870-c6271eea2158 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:16.929064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f3d10-013d-42f2-9870-c6271eea2158 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:16.929535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e43f3d10-013d-42f2-9870-c6271eea2158 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:16.931110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 120/237] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:16.937269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:16.937562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:16.937850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:16.969982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:16.971203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:16.994933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03a64146-5a9f-4ca7-bc5c-078d3844db71 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.053830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:17.068041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.068851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.069137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.069931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.095624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:17.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:17.106380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:17.107596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 118/238] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:16 2022] GET /identity/v3/roles/reader => generated 84 bytes in 235 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:17.114606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 121/239] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:17.118268 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.146390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.212815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:17.257657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c69bbc-b3a0-4246-82bc-f00bd72d654c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.267082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de4560ba-4ce4-43ca-ac10-969cdad16d3c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.293818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.294872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.295352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.295823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.296808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:17.297308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:17.299224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e76d0d2-6404-4418-ab24-854c91845d04 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:17.307679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab39eef1-45c9-4793-ba79-6460b74461e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.308522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39eef1-45c9-4793-ba79-6460b74461e8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.308810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39eef1-45c9-4793-ba79-6460b74461e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.309088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab39eef1-45c9-4793-ba79-6460b74461e8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.310314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39eef1-45c9-4793-ba79-6460b74461e8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:17.311749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab39eef1-45c9-4793-ba79-6460b74461e8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:17.313630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 122/240] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:17.314595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657987a-bf57-4dc1-91ba-e82b4a1b64fa None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.314939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657987a-bf57-4dc1-91ba-e82b4a1b64fa None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.315230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e657987a-bf57-4dc1-91ba-e82b4a1b64fa None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.321928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 123/241] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:17.323172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 119/242] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:17 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:17.323282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657987a-bf57-4dc1-91ba-e82b4a1b64fa None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.323573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.323894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.327305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.348551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:17.350061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:17.417973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:17.437988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e657987a-bf57-4dc1-91ba-e82b4a1b64fa None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.447789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 120/243] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:17.448950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.478252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:17.497807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.498631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.499053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.499437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.500644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:17.501859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:17.563353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b038518d-e973-4821-970f-64d0f79ff2c4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.583182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 124/244] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:17 2022] POST /identity/v3/users => generated 293 bytes in 261 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:17.584448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.604682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.605572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.605852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.606129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] PATH_INFO: `/v3/users/b2a1963c78484b9ca0c204b506ecaf4f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.633123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] RBAC: Authorizing `identity:get_user(user_id=b2a1963c78484b9ca0c204b506ecaf4f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:17.634640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:17.645333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 121/245] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:17 2022] GET /identity/v3/users/b2a1963c78484b9ca0c204b506ecaf4f => generated 297 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:17.654229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.734345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:17.796169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ce99e92-b290-45c0-a95b-797ccfaa36b3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.832665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-659eb77b-cfb9-45af-88f6-3205527683a5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.833148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.833667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.833957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.834235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.846139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:17.854031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:17.854889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:17.855652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 125/246] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:17 2022] GET /identity/v3/roles/member => generated 84 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:17.861311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:17.876940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:17.877920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:17.878507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:17.878987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:17.896141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:17.957166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:17.958957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:17.967094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bfaf6d-2fcf-4982-bd0e-44f1906511f7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:17.999232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:18.002553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:18.003006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.003234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.003511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.004382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:18.004695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:18.006164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:18.015676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 126/247] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:17 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:18.021410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:18.066387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 122/248] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:18.076384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:18.137475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:18.145243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011d002-ba6a-45cd-a258-87e8c4e9396a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:18.184316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:18.185531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.185987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f14d92ff-f395-4b99-9806-c4a79f182e05 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:18.186340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.186817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] PATH_INFO: `/v3/users/24f6f9cbeccc48c5acfcbe4611fd3917` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.188539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] RBAC: Authorizing `identity:get_user(user_id=24f6f9cbeccc48c5acfcbe4611fd3917)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:18.190341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:18.195149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 127/249] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:18 2022] GET /identity/v3/users/24f6f9cbeccc48c5acfcbe4611fd3917 => generated 297 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:18.199244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:18.225249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:18.226651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.226932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.227216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.228486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:18.230178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:18.231907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 123/250] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:18 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:18.244426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:18.254031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:18.314257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f239e23f-f468-4d2c-b0aa-ab88c67ac873 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:18.350324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:18.351770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.352146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.352424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.353646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:18.355106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:18.356601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e3f9dfa-aab0-44e9-a401-c633f89ce112 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:18.359984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 128/251] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:18 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:18.362894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:18.397361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:18.398147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:18.398484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.399596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.400072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.401056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] RBAC: Adding query filter params (role.id=30c1267a61504942b5b1382cc1d47901, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=b2a1963c78484b9ca0c204b506ecaf4f) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:18.401513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:18.403092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ccc7bfb-223d-4f5d-b206-5c415d714979 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:18.409492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 124/252] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:18 2022] GET /identity/v3/role_assignments?user.id=b2a1963c78484b9ca0c204b506ecaf4f&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=30c1267a61504942b5b1382cc1d47901 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:18.481276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22d54ebf-1d14-461e-aefd-348512fb2b96 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:18.519095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863e8193-578d-4f71-85fc-3bc4a6133a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:18.520034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e8193-578d-4f71-85fc-3bc4a6133a03 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.520326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e8193-578d-4f71-85fc-3bc4a6133a03 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.520605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863e8193-578d-4f71-85fc-3bc4a6133a03 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/24f6f9cbeccc48c5acfcbe4611fd3917/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.522687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e8193-578d-4f71-85fc-3bc4a6133a03 None admin] RBAC: Authorizing `identity:create_grant(project_id=90ef71bb81154cd8856ce17bcf6eb121, user_id=24f6f9cbeccc48c5acfcbe4611fd3917, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:18.524173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863e8193-578d-4f71-85fc-3bc4a6133a03 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:18.609682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 129/253] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:18 2022] PUT /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/24f6f9cbeccc48c5acfcbe4611fd3917/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:37:18.650211 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99b3a0-2345-464e-8d54-980200853dc4 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.650832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99b3a0-2345-464e-8d54-980200853dc4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.651383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be99b3a0-2345-464e-8d54-980200853dc4 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.653126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 125/254] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:18.666276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:18.666565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:18.666851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:18.687852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:18.689009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:18.778865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 130/255] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:18.825530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99b3a0-2345-464e-8d54-980200853dc4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:18.919984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:18.962748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be99b3a0-2345-464e-8d54-980200853dc4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:19.004880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f989eb-e75e-4b0c-b226-3f364ddd06fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:19.006035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f989eb-e75e-4b0c-b226-3f364ddd06fb None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.006512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f989eb-e75e-4b0c-b226-3f364ddd06fb None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.007530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f989eb-e75e-4b0c-b226-3f364ddd06fb None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.024022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8f989eb-e75e-4b0c-b226-3f364ddd06fb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-a8f989eb-e75e-4b0c-b226-3f364ddd06fb None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:19.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:19.032901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:19.033674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:19.036343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 126/256] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:19.042073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:19.077024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:19.110087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae13ae-e21e-4604-8ca5-f3b22c5847f6 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.111959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae13ae-e21e-4604-8ca5-f3b22c5847f6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.112458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cae13ae-e21e-4604-8ca5-f3b22c5847f6 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.114714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 127/257] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:19 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:19.121186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.122364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.123652 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.148002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:19.149217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:19.275701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b6a6ba6-756a-459d-864a-0bc70a713ace None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:19.344851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 128/258] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:19.353971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:19.354702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.354994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.355272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.356060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:19.356372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:19.357801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:19.370798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 131/259] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 326 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:19.372868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:19.384965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:19.486396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:19.562622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1f6b10-bb7c-467a-a3e8-aa166e04ec96 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:19.577496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:19.613697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad99fd49-1293-4fef-b91b-25ffa7a4fad8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:19.640716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:19.642138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.642425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.642702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] PATH_INFO: `/v3/users/9a71fc60dc954122aa77370a9de40bef` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.644821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] RBAC: Authorizing `identity:get_user(user_id=9a71fc60dc954122aa77370a9de40bef)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:19.646280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:19.653674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 129/260] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:19 2022] GET /identity/v3/users/9a71fc60dc954122aa77370a9de40bef => generated 283 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:19.658554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:19.681655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:19.682489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.682783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.683068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.709787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:19.721235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:19.722131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:19.722902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:19.727395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 132/261] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:19 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 343 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:19.732728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:19.853830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa784aa2-c0c7-475a-a6fb-884d5f37c7df None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:19.897101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:19.915072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:19.915739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:19.916135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:19.916419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:19.998636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0bde104-4278-4536-92c7-956682400e56 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:20.057898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:20.059351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:20.059829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:20.060322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:20.061290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:20.061893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.063463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.064679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.066284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.073047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 133/262] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:19 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 341 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:20.080834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:20.084732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:20.211729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73b65c9-8348-4ea3-968e-63923a86bf42 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:20.246465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:20.247720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:20.248228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:20.248718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] PATH_INFO: `/v3/users/9a71fc60dc954122aa77370a9de40bef` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:20.250122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] RBAC: Authorizing `identity:get_user(user_id=9a71fc60dc954122aa77370a9de40bef)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.251769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.253965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 134/263] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:20 2022] GET /identity/v3/users/9a71fc60dc954122aa77370a9de40bef => generated 283 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:20.258936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:20.277717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 130/264] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:19 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 619 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:20.283158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:20.340233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:20.362188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:20.394040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c2bf9df-ddbe-461c-a2c3-77b90bdd309b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:20.439835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:20.440527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:20.440820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:20.441103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:20.441522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf96caa1-fae3-43cd-a185-0efc839a0bfa None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:20.476625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:20.477727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:20.478028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:20.478305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:20.479504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.481008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.483023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 131/265] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:20 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:20.488766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:20.558289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.559767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.629793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:20.637737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b2786f-d612-4f5f-a0b6-79692e742f7b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:20.680471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:20.727898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 135/266] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 469 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:20.735704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:20.757824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:20.758509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:20.758795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:20.759073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:20.764030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=9a71fc60dc954122aa77370a9de40bef) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:20.764468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.765983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.782570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 132/267] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:20 2022] GET /identity/v3/role_assignments?user.id=9a71fc60dc954122aa77370a9de40bef&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 294 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:20.889951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27644c82-77fd-439d-b915-314e89e7ad58 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:20.939844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9fb3115-8083-488b-942f-6fcf3f02c97a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:20.940755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3115-8083-488b-942f-6fcf3f02c97a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:20.941040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3115-8083-488b-942f-6fcf3f02c97a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:20.941319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9fb3115-8083-488b-942f-6fcf3f02c97a None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:20.948867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb3115-8083-488b-942f-6fcf3f02c97a None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:20.950390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9fb3115-8083-488b-942f-6fcf3f02c97a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:20.952273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 136/268] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:20 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:20.958143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.095644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d5dc8-b0ef-4707-8d7c-d228842fa09c None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.096173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d5dc8-b0ef-4707-8d7c-d228842fa09c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.096173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d49d5dc8-b0ef-4707-8d7c-d228842fa09c None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.097805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 137/269] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:21.120069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711b159-4a42-4d01-ab3c-58cf81a4b93c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.121562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711b159-4a42-4d01-ab3c-58cf81a4b93c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.121562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3711b159-4a42-4d01-ab3c-58cf81a4b93c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.136400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a7e49d-4010-47be-82fe-fd77d2f0f498 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:21.145020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3711b159-4a42-4d01-ab3c-58cf81a4b93c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:21.146238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3711b159-4a42-4d01-ab3c-58cf81a4b93c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:21.190871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:21.191542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.191824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.192127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.193072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] RBAC: Adding query filter params (role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=9a71fc60dc954122aa77370a9de40bef) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:21.193391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:21.195464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:21.204282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 133/270] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:20 2022] GET /identity/v3/role_assignments?user.id=9a71fc60dc954122aa77370a9de40bef&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c4622473a5824ed7876be10e3da526ca => generated 608 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:21.207962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:21.261245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 138/271] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:21.299516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.332072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:21.365527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cb6aa-0d82-4aa4-84fa-2d13a01320e8 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.365827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cb6aa-0d82-4aa4-84fa-2d13a01320e8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.366031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-680cb6aa-0d82-4aa4-84fa-2d13a01320e8 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.368292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 139/272] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:21.376742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bacfc-65f2-4a05-932e-3df5db5c4686 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.377459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bacfc-65f2-4a05-932e-3df5db5c4686 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.377937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022bacfc-65f2-4a05-932e-3df5db5c4686 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.410523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-022bacfc-65f2-4a05-932e-3df5db5c4686 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:21.412235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-022bacfc-65f2-4a05-932e-3df5db5c4686 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:21.426912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a7602d-c9e8-49fd-ae16-94b8d2f4ab6f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:21.469411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:21.470798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.471324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.471943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.484588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:21.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:21.495042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:21.495808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:21.500520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 140/274] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:21.500775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 134/274] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:21 2022] GET /identity/v3/roles/member => generated 84 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:21.501872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389d04-9e9f-4e17-b6cb-a25e9a37d8d6 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.502175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389d04-9e9f-4e17-b6cb-a25e9a37d8d6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.502446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0389d04-9e9f-4e17-b6cb-a25e9a37d8d6 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.504272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 141/275] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:21.505991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.519275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.519908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.520398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.544494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:21.545700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:21.613365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:21.620304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 142/276] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:21.621163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.639243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77d9e0ee-bd88-44ca-a32e-78ca491ee21a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:21.667464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:21.676907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:21.677835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.678707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.678707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.679658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:21.680005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:21.681412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:21.690340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 135/277] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:21 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:21.691142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.752002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-937a0bd9-48d4-44e9-848f-c60993c04598 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:21.788893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:21.789686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.789966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.790235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.791036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b34a724b-e41b-4169-9703-9270e8dd2b1b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:21.791384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:21.792985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:21.799118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 143/278] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:21.799118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.831299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:21.832127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.832415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.832688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.843450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:21.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:21.852860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:21.853628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 136/279] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:21 2022] GET /identity/v3/roles/reader => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:21.854262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.866503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:21.903159 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:21.906923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e1326f1-7f12-48bb-878e-f9b7712538f7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:21.943810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:21.944748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:21.945333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:21.945617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] PATH_INFO: `/v3/users/85562a9ddb654e6db4992d3248ce73c8` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:21.959464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] RBAC: Authorizing `identity:get_user(user_id=85562a9ddb654e6db4992d3248ce73c8)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:21.961478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:21.963420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 144/280] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:21 2022] GET /identity/v3/users/85562a9ddb654e6db4992d3248ce73c8 => generated 293 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:21.968330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:21.974177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e315b7-9290-4742-8179-89a1f76e3a35 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.017460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.018185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.018452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.018715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.019458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.020997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.074966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b50adcc-8f43-4f27-8aef-78a574d6446b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.123897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 137/281] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:21 2022] POST /identity/v3/users => generated 293 bytes in 256 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:22.124096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.124561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.124638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.124638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.124638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.124638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:22.124638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.124638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58ee68e0-a805-4bb7-99c4-1d38f6090a19 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.124964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 145/282] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:21 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:22.126883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cc6b7-b8ed-4572-8645-a72ed63a778d None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.127902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cc6b7-b8ed-4572-8645-a72ed63a778d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.127902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8cc6b7-b8ed-4572-8645-a72ed63a778d None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.130118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 146/283] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:22 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:22.131219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cc6b7-b8ed-4572-8645-a72ed63a778d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.159032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:22.185127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:22.229736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa7fb2d-1bdf-4427-be10-3c82607ff311 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.264693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8cc6b7-b8ed-4572-8645-a72ed63a778d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.269640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.271100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.271392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.271677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.273114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, user.id=85562a9ddb654e6db4992d3248ce73c8, scope.system=all) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:22.273435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.274904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d20f4c8-6756-4e72-8933-6951b5ab6cc1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.280828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 138/284] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/role_assignments?user.id=85562a9ddb654e6db4992d3248ce73c8&scope.system=all&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 226 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:22.281781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.282076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.282360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.306962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:22.308265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:22.315274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.316231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.316499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.316780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] PATH_INFO: `/v3/users/b2a1963c78484b9ca0c204b506ecaf4f` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.318683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] RBAC: Authorizing `identity:get_user(user_id=b2a1963c78484b9ca0c204b506ecaf4f)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.321286 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.324062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 147/285] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/users/b2a1963c78484b9ca0c204b506ecaf4f => generated 297 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:22.329000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.392670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 139/286] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:22 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:22.419727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.437529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:22.454622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee0435ee-dc5c-44f1-8ee7-9e4273d2239e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.499176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:22.512399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.513298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.513587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.513866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.516107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.517593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.519640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 148/287] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:22.526323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.553204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d246bc9-5f13-48a1-a1b5-f3d69f5e634f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.601086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.602153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.602443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.602725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.616790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:22.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:22.628504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:22.629282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 140/288] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/roles/member => generated 84 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:22.635286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.651498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2958faa-b6e2-4240-9a44-6104ac1da71b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.717203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.718427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.718852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.719299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/b2a1963c78484b9ca0c204b506ecaf4f/roles/30c1267a61504942b5b1382cc1d47901` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.721500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] RBAC: Authorizing `identity:create_grant(project_id=90ef71bb81154cd8856ce17bcf6eb121, user_id=b2a1963c78484b9ca0c204b506ecaf4f, role_id=30c1267a61504942b5b1382cc1d47901)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.723077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.753488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af18b569-2342-4744-aea8-a6a0d663140f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.771130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:22.790442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.791134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.791423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.791704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.792500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:22.792814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.794220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.803426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 141/289] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:22.809395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.814039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 149/290] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:37:22 2022] PUT /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/users/b2a1963c78484b9ca0c204b506ecaf4f/roles/30c1267a61504942b5b1382cc1d47901 => generated 0 bytes in 288 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:37:22.921340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66fbe0c9-61c4-490d-8db6-b2968ee152d7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:22.959845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:22.960777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:22.961065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:22.961344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] PATH_INFO: `/v3/users/24f6f9cbeccc48c5acfcbe4611fd3917` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:22.962908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] RBAC: Authorizing `identity:get_user(user_id=24f6f9cbeccc48c5acfcbe4611fd3917)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:22.964443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:22.966518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 142/291] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/users/24f6f9cbeccc48c5acfcbe4611fd3917 => generated 297 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:22.972132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:22.996250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:23.110944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75f4edef-09ea-4a1c-beee-721b1f6bdc06 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:23.149981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:23.150747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:23.150930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:23.151199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:23.239506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:23.240943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:23.259999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:23.368943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 150/292] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 397 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:23.374241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:23.447510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:23.501945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe2b3e-3a87-46e7-8955-6406956bd0d2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:23.537717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d289f6d1-2014-47f2-8a03-89ec6f509330 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:23.539138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f6d1-2014-47f2-8a03-89ec6f509330 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:23.539425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f6d1-2014-47f2-8a03-89ec6f509330 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:23.539709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d289f6d1-2014-47f2-8a03-89ec6f509330 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:23.540958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f6d1-2014-47f2-8a03-89ec6f509330 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:23.542402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d289f6d1-2014-47f2-8a03-89ec6f509330 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:23.544447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 143/293] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:23 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:23.550398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:23.676082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-996cf80f-1f50-469d-9d42-d7958ce3e9ae None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:23.697552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:23.718670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:23.719485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:23.719851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:23.720240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:23.721169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=24f6f9cbeccc48c5acfcbe4611fd3917) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:23.721564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:23.723100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af8f0f0-3460-4fc1-935c-7c6c81b05b2a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:23.730445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 151/294] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:23 2022] GET /identity/v3/role_assignments?user.id=24f6f9cbeccc48c5acfcbe4611fd3917&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:24.724431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35cd6-4db9-44c5-b930-0b94021df36d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:24.725424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35cd6-4db9-44c5-b930-0b94021df36d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:24.725927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a35cd6-4db9-44c5-b930-0b94021df36d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:24.727777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 144/295] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:24.732763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8be213-e512-4d95-9ae6-bb77a84d5f75 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:24.733586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8be213-e512-4d95-9ae6-bb77a84d5f75 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:24.734067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d8be213-e512-4d95-9ae6-bb77a84d5f75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:24.751512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2d8be213-e512-4d95-9ae6-bb77a84d5f75 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:24.753061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2d8be213-e512-4d95-9ae6-bb77a84d5f75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:24.827268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 152/296] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:24.850635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35cd6-4db9-44c5-b930-0b94021df36d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:24.860166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b082040-ce0a-475d-8ea8-b4177c539acb None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:24.860466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b082040-ce0a-475d-8ea8-b4177c539acb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:24.860745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b082040-ce0a-475d-8ea8-b4177c539acb None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:24.862369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 153/297] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:24.867975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea0766-0e86-42ab-b8b9-dfe077429834 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:24.868261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea0766-0e86-42ab-b8b9-dfe077429834 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:24.868543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6ea0766-0e86-42ab-b8b9-dfe077429834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:24.885924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-f6ea0766-0e86-42ab-b8b9-dfe077429834 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:24.887152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f6ea0766-0e86-42ab-b8b9-dfe077429834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:24.957580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 154/298] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:24.965581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7df4e-e46e-4416-b741-973dbad92b12 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:24.966097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7df4e-e46e-4416-b741-973dbad92b12 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:24.966388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b7df4e-e46e-4416-b741-973dbad92b12 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:24.967783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a35cd6-4db9-44c5-b930-0b94021df36d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:24.967951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 155/299] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:24 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:24.973399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:24.973688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:24.973963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:24.990869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:24.992632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:24.992792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.018796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.019609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.020158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.020261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.023731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.030794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:25.039489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:25.040592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:25.041351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:25.047914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 145/300] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:24 2022] GET /identity/v3/roles/member => generated 84 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:25.047914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.066163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 156/301] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:25.067381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.167567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f881ac9-160b-46bc-827a-c766e5b6d373 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.170737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad43f26e-e552-47b3-b7a9-721a8f0a7ac6 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.252684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.253279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.253628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.253909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.254689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:25.254967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.255085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.255763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.256080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.256364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.256475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.257533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.258951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.260816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 146/302] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:25.269903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 157/303] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:25.270061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.270147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.308751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.317497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.378952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cad8902e-1757-42a1-ae61-6749145344f4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.385657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad834a22-328d-4396-80d6-75394ebad3f4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.414028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.414831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.415112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.415391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.419822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1d76d02-7c1b-49b5-8caf-14cb90512c03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.420530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76d02-7c1b-49b5-8caf-14cb90512c03 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.420818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76d02-7c1b-49b5-8caf-14cb90512c03 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.421099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1d76d02-7c1b-49b5-8caf-14cb90512c03 None admin] PATH_INFO: `/v3/groups` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.421789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76d02-7c1b-49b5-8caf-14cb90512c03 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.423241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1d76d02-7c1b-49b5-8caf-14cb90512c03 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.425402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:25.433172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:25.434233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:25.434995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 147/304] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:25.435624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.499705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 158/305] 10.222.0.40 () {60 vars in 1134 bytes} [Thu Jun 16 11:37:25 2022] POST /identity/v3/groups => generated 229 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:25.501047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b0ff3-fb07-4e36-9f51-4e686073718a None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.501342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b0ff3-fb07-4e36-9f51-4e686073718a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.501891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a7b0ff3-fb07-4e36-9f51-4e686073718a None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.504074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 159/306] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:25.504074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b0ff3-fb07-4e36-9f51-4e686073718a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.546939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f492f61b-38da-4883-8219-4903bb951553 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.548610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.585779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be1dd3ad-e9e8-4dab-be44-c7296fc6cd0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.586789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd3ad-e9e8-4dab-be44-c7296fc6cd0a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.587406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd3ad-e9e8-4dab-be44-c7296fc6cd0a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.587757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be1dd3ad-e9e8-4dab-be44-c7296fc6cd0a None admin] PATH_INFO: `/v3/users/85562a9ddb654e6db4992d3248ce73c8` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.589189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dd3ad-e9e8-4dab-be44-c7296fc6cd0a None admin] RBAC: Authorizing `identity:get_user(user_id=85562a9ddb654e6db4992d3248ce73c8)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.590710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be1dd3ad-e9e8-4dab-be44-c7296fc6cd0a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.592671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 148/307] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/users/85562a9ddb654e6db4992d3248ce73c8 => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:25.593809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.594108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.594392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.612282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:25.896839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:25.898631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.898631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a7b0ff3-fb07-4e36-9f51-4e686073718a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.899065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 149/308] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:25.899151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.899250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.899337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.899337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.899337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.899337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] RBAC: Adding query filter params (name=reader) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:25.899337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.899337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.900010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 160/309] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:25.900010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.900182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:25.900182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36342533-d5d9-4951-b430-6e1d34736e1c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.900335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1798ca29-dca3-4685-8beb-bfb4d9dae198 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.900423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.900423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.900423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.900423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] PATH_INFO: `/v3/system/users/85562a9ddb654e6db4992d3248ce73c8/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.900719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=85562a9ddb654e6db4992d3248ce73c8, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.900719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.900874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:25.900874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:25.900874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:25.900874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] PATH_INFO: `/v3/users/cdf7f806c890415bbc8f1280abfd838f` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:25.901343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] RBAC: Authorizing `identity:get_user(user_id=cdf7f806c890415bbc8f1280abfd838f)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:25.901343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:25.901343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 161/310] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/users/cdf7f806c890415bbc8f1280abfd838f => generated 293 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:25.901343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:25.974230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab54d24-d2d9-4f41-a1dd-c235a3ba2688 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:25.979903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 150/311] 10.222.0.40 () {58 vars in 1334 bytes} [Thu Jun 16 11:37:25 2022] PUT /identity/v3/system/users/85562a9ddb654e6db4992d3248ce73c8/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 283 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:25.979903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:26.038397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:26.039398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:26.039694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:26.040133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:26.053003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:26.061527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:26.062534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:26.063319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 162/312] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/roles/reader => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:26.064747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:26.068743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:26.124941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b31760bc-c767-4cee-894a-8e55716d03df None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:26.157955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:26.173082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:26.177054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:26.177054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:26.177054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:26.177054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] RBAC: Adding query filter params (role.id=30c1267a61504942b5b1382cc1d47901, user.id=cdf7f806c890415bbc8f1280abfd838f, scope.system=all) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:26.177054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:26.177854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:26.184786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 151/313] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 11:37:25 2022] GET /identity/v3/role_assignments?user.id=cdf7f806c890415bbc8f1280abfd838f&scope.system=all&role.id=30c1267a61504942b5b1382cc1d47901 => generated 226 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:26.195467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bc36bb2-4000-420e-96e5-042be313f375 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:26.235395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:26.236111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:26.236397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:26.236676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:26.237507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] RBAC: Adding query filter params (name=reader) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:26.237865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:26.239307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:26.250344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 163/314] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:26 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:26.255618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:26.328608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:26.368526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a79ea7-ab9d-4a09-a4ea-4cf87b45ddfe None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:26.410068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:26.411305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:26.411809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:26.412315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] PATH_INFO: `/v3/users/b2a1963c78484b9ca0c204b506ecaf4f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:26.413766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] RBAC: Authorizing `identity:get_user(user_id=b2a1963c78484b9ca0c204b506ecaf4f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:26.415419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:26.417601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 152/315] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:26 2022] GET /identity/v3/users/b2a1963c78484b9ca0c204b506ecaf4f => generated 297 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:26.425392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:26.530505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1757c35-37c8-44a0-9680-5b2a319f609a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:26.564386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:26.570568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:26.571228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:26.571513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:26.571787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:26.642348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:26.643813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:26.750867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 164/316] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:26.757119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:26.854746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:26.861069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73c54de0-231c-480d-8ab2-2c9e874c231a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:26.896454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66def51d-0e64-41a0-99b8-58fcde5f222c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:26.897347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66def51d-0e64-41a0-99b8-58fcde5f222c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:26.897882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66def51d-0e64-41a0-99b8-58fcde5f222c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:26.898185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66def51d-0e64-41a0-99b8-58fcde5f222c None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:26.899381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66def51d-0e64-41a0-99b8-58fcde5f222c None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:26.900863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66def51d-0e64-41a0-99b8-58fcde5f222c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:26.907449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 153/317] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:26 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:26.908839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:26.940585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:27.013453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c74929-1051-4783-b752-58f71066b776 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:27.047585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:27.048270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:27.048554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:27.048830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:27.049656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] RBAC: Adding query filter params (role.id=30c1267a61504942b5b1382cc1d47901, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121, user.id=b2a1963c78484b9ca0c204b506ecaf4f) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:27.049968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:27.051400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf165847-a6a3-4bc0-a8a9-de85cc3f8b9c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:27.058065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 165/318] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:26 2022] GET /identity/v3/role_assignments?user.id=b2a1963c78484b9ca0c204b506ecaf4f&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=30c1267a61504942b5b1382cc1d47901 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:27.749666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489f56-e584-44c0-8ce6-3489f970d725 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:27.749995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489f56-e584-44c0-8ce6-3489f970d725 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:27.750230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2489f56-e584-44c0-8ce6-3489f970d725 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:27.751748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 154/319] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:27.756660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:27.756955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:27.757233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:27.774059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:27.775231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:27.841510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 166/320] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:27.863660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489f56-e584-44c0-8ce6-3489f970d725 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:27.963730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2489f56-e584-44c0-8ce6-3489f970d725 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:27.996771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6209e6ce-7535-4eb5-a7b4-49bfd994f002 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:27.997568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e6ce-7535-4eb5-a7b4-49bfd994f002 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:27.997854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e6ce-7535-4eb5-a7b4-49bfd994f002 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:27.998125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6209e6ce-7535-4eb5-a7b4-49bfd994f002 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:27.999268 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209e6ce-7535-4eb5-a7b4-49bfd994f002 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:28.000738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6209e6ce-7535-4eb5-a7b4-49bfd994f002 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:28.002499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 155/321] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:27 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:28.008273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.034859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:28.104610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3932aa-9f0c-45bf-9a36-a304abdadd1c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:28.142582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:28.143336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.143622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.143936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] PATH_INFO: `/v3/groups` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.144641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:28.146137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:28.176130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c12fa-365d-4a17-b2cb-31389be70923 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.176414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c12fa-365d-4a17-b2cb-31389be70923 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.176689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5c12fa-365d-4a17-b2cb-31389be70923 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.178042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 156/322] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:28.182782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503236f-9974-4888-b03c-ff9fe54a0cd3 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.183069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503236f-9974-4888-b03c-ff9fe54a0cd3 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.183345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7503236f-9974-4888-b03c-ff9fe54a0cd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.216332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7503236f-9974-4888-b03c-ff9fe54a0cd3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:28.217430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7503236f-9974-4888-b03c-ff9fe54a0cd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:28.241166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 167/323] 10.222.0.40 () {60 vars in 1134 bytes} [Thu Jun 16 11:37:28 2022] POST /identity/v3/groups => generated 226 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:37:28.247905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:28.296625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 157/324] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:28.320098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.336237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3b25d-f2f4-4ed8-b98a-81602f14626e None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.336553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3b25d-f2f4-4ed8-b98a-81602f14626e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.336799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fa3b25d-f2f4-4ed8-b98a-81602f14626e None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.338461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 158/325] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:28.345232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.345586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.345813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.363727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:28.364988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:28.421079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:28.441616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76767aad-70e2-40be-8f6b-8fe39c891947 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:28.442380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 159/326] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:28 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:28.465734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.477710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:28.478446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.478725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.479008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.492096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:28.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:28.500618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:28.501834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 168/327] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:28.508600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.581070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:28.591655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-489ec293-7907-4030-bbca-6f7f03a505cc None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:28.632074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:28.632861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9892cd8d-411d-46dc-9bfc-71735f8a62e3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:28.632965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.633159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.633439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.647034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:28.659493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:28.660431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:28.661207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:28.661989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 160/328] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/roles/reader => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:28.668138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.678741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:28.679437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.679732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.680049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.680878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:28.681201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:28.682612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:28.695651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 169/329] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:28.701876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.715444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:28.821585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c199c6c-1793-4342-9f23-2b5b91fd85f4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:28.826027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c57084a-7ab9-483a-9acc-f190404ea1c7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:28.866551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:28.867438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.867724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.868050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] PATH_INFO: `/v3/users/85562a9ddb654e6db4992d3248ce73c8` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.868271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:28.869246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:28.869350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] RBAC: Authorizing `identity:get_user(user_id=85562a9ddb654e6db4992d3248ce73c8)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:28.869828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:28.870316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:28.870780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:28.871488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] RBAC: Adding query filter params (name=reader) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:28.872033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:28.872936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 170/330] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/users/85562a9ddb654e6db4992d3248ce73c8 => generated 293 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:28.873822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:28.880863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.883601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 161/331] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:28.889797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:28.910019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:28.984844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:29.015706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faeff0eb-3eb7-4976-86be-acf568b450d1 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:29.019905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d7f3f1e-b959-4a3d-a442-68b02482fa5e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:29.062042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:29.062867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:29.063100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:29.063395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:29.064319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] RBAC: Adding query filter params (role.id=c2d46bdeda48498b8c04ea76027be796, user.id=85562a9ddb654e6db4992d3248ce73c8, scope.system=all) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:29.064630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:29.066086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acda5ffd-163a-4612-baa7-c36cac6280ff None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:29.068427 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:29.070144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:29.070633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:29.071109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] PATH_INFO: `/v3/users/cdf7f806c890415bbc8f1280abfd838f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:29.073148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] RBAC: Authorizing `identity:get_user(user_id=cdf7f806c890415bbc8f1280abfd838f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:29.075239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:29.078025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 162/332] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/users/cdf7f806c890415bbc8f1280abfd838f => generated 293 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:29.081702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 171/333] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 11:37:28 2022] GET /identity/v3/role_assignments?user.id=85562a9ddb654e6db4992d3248ce73c8&scope.system=all&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 510 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:29.083887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:29.181098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:29.201626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7d8d7ca-e964-4808-b221-c2d58f8d96a3 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:29.240394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b78af1c7-2d29-4115-90bb-05cab0835479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:29.241712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78af1c7-2d29-4115-90bb-05cab0835479 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:29.242198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78af1c7-2d29-4115-90bb-05cab0835479 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:29.242708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b78af1c7-2d29-4115-90bb-05cab0835479 None admin] PATH_INFO: `/v3/system/users/cdf7f806c890415bbc8f1280abfd838f/roles/30c1267a61504942b5b1382cc1d47901` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:29.244466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78af1c7-2d29-4115-90bb-05cab0835479 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=cdf7f806c890415bbc8f1280abfd838f, role_id=30c1267a61504942b5b1382cc1d47901)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:29.246125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b78af1c7-2d29-4115-90bb-05cab0835479 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:29.332992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 163/334] 10.222.0.40 () {58 vars in 1334 bytes} [Thu Jun 16 11:37:29 2022] PUT /identity/v3/system/users/cdf7f806c890415bbc8f1280abfd838f/roles/30c1267a61504942b5b1382cc1d47901 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:30.784581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f2e3-5301-4e0b-9153-d2426e5d799e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:30.785119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f2e3-5301-4e0b-9153-d2426e5d799e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:30.785119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9971f2e3-5301-4e0b-9153-d2426e5d799e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:30.786744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 172/335] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:30.792069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:30.792461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:30.792670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:30.811276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:30.812828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:30.900890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 164/336] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:30.936734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f2e3-5301-4e0b-9153-d2426e5d799e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:31.087896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9971f2e3-5301-4e0b-9153-d2426e5d799e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:31.134405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:31.135216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.135502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.135782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.137780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:31.152772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:31.160590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:31.161512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:31.162302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 173/337] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:30 2022] GET /identity/v3/roles/member => generated 84 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:31.168742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:31.313216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:31.342499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccbdbb7d-acd2-4d4c-bc48-e25c18ddff6d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:31.395287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:31.395998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.396291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.396639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.397997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:31.398317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:31.399717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5de7c83-db77-4e60-8b28-3b5fdf605d23 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:31.409452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 165/338] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:31 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:31.415212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:31.448381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474f0c-6b72-40d8-bde4-d2f1029e0dd5 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.449111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474f0c-6b72-40d8-bde4-d2f1029e0dd5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.449403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b474f0c-6b72-40d8-bde4-d2f1029e0dd5 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.451015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 166/339] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:31.463006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.463863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.464179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.497500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:31.498727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:31.608860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 167/340] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:31.627462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c710f4d3-4625-4af9-8438-a9a9495ca050 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:31.682222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29afb6f9-6bb3-4070-8ed3-b7a90c724847 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:31.682906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afb6f9-6bb3-4070-8ed3-b7a90c724847 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.684530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:31.696256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afb6f9-6bb3-4070-8ed3-b7a90c724847 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.696592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29afb6f9-6bb3-4070-8ed3-b7a90c724847 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.775635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:31.818418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61c71e8-97a7-42d8-a61c-52da5aef2d4d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:31.833787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:31.847484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afb6f9-6bb3-4070-8ed3-b7a90c724847 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:31.854406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29afb6f9-6bb3-4070-8ed3-b7a90c724847 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:31.867223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f2b5082-587f-408c-ba0e-73471bd1d37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:31.868298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5082-587f-408c-ba0e-73471bd1d37d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.868733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5082-587f-408c-ba0e-73471bd1d37d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.869178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f2b5082-587f-408c-ba0e-73471bd1d37d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.890443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f2b5082-587f-408c-ba0e-73471bd1d37d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-6f2b5082-587f-408c-ba0e-73471bd1d37d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:31.905976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:31.907079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:31.907992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:31.913935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 168/341] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:31.918628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ab846-cec9-44e2-b9ae-ba8553b112df None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:31.922593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ab846-cec9-44e2-b9ae-ba8553b112df None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:31.923067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ab846-cec9-44e2-b9ae-ba8553b112df None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:31.924634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 169/342] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:31.926577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ab846-cec9-44e2-b9ae-ba8553b112df None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.062686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 174/343] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:31 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 648 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:32.067929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.070030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.070463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.105823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:32.108473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:32.139387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ab846-cec9-44e2-b9ae-ba8553b112df None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:32.176544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:32.190037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:32.191356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.191893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.195024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.196082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:32.196606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:32.198228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:32.217500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 170/344] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:31 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 291 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:32.219935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.262461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 175/345] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:32.263787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.314643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:32.415787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f442577-36d9-4ee7-b7f6-807136ed2840 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:32.467850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe0dc62-032c-4305-8f45-318111e788a1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:32.499829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:32.501183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.501716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.502199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.505070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:32.506936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:32.512193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 171/346] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:32 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:32.513442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.541659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c119d275-7efc-4a18-a794-d13c9c19b3b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:32.542718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119d275-7efc-4a18-a794-d13c9c19b3b0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.543249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119d275-7efc-4a18-a794-d13c9c19b3b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.543791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c119d275-7efc-4a18-a794-d13c9c19b3b0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.609055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:32.635516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119d275-7efc-4a18-a794-d13c9c19b3b0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:32.638191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c119d275-7efc-4a18-a794-d13c9c19b3b0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:32.663734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ec5363-1269-4052-b07c-961b127e6d6f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:32.664924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:32.713601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:32.714343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.714631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.714911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.728255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:32.737150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:32.738091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:32.739011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:32.740250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 172/347] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:32.741276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.773881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 176/348] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:32 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 510 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:32.775042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17454f-d644-4232-88fd-76e4ec7539e3 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.775338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17454f-d644-4232-88fd-76e4ec7539e3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.775620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a17454f-d644-4232-88fd-76e4ec7539e3 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.777162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 177/349] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:32.782153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60690-5bdb-40a7-9f5c-2acd5b9930f0 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.782726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60690-5bdb-40a7-9f5c-2acd5b9930f0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.783022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9f60690-5bdb-40a7-9f5c-2acd5b9930f0 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.786625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 178/350] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:32.786625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60690-5bdb-40a7-9f5c-2acd5b9930f0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.872595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee07a95a-0476-45bb-8631-83ce49d20277 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:32.901160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:32.921283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9f60690-5bdb-40a7-9f5c-2acd5b9930f0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:32.926767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:32.926767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.926767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.926767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.930711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:32.932212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:32.934071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 173/351] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:32 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:32.934681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:32.954989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:32.975178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:32.975178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.975468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.975645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:32.976750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:32.977133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:32.979329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d76d550-8abe-4e0d-afeb-690b8f7f6b0c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:32.989235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 179/352] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:32.991158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33114c51-178e-4f68-ba46-bd6f2a9f398d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:32.991450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33114c51-178e-4f68-ba46-bd6f2a9f398d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:32.991786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33114c51-178e-4f68-ba46-bd6f2a9f398d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.014440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-33114c51-178e-4f68-ba46-bd6f2a9f398d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:33.015668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-33114c51-178e-4f68-ba46-bd6f2a9f398d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:33.077551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e9ee84d-89db-4510-93d1-f50131d04cce None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.103276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 180/353] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:33.105215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.105215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.105215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.123102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.124020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.124301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.124623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.125761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:33.127195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:33.128966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 174/354] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:32 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:33.129312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:33.130599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:33.132937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.187190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:33.215498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 181/355] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:33.216137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.248988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4865371-fdbc-4c40-bbef-2ccc157804a6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.272903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:33.292493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.293170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.293449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.293725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.294563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=85416ef9b7144e4889bdac4457d53735) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:33.294878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:33.297023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe20bdbd-2cd6-438e-bb88-70e43d4c96b0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:33.306583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 175/356] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 260 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:33.308779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101653f-4687-4db7-ad72-c63a2f35dfa2 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.308779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101653f-4687-4db7-ad72-c63a2f35dfa2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.308779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9101653f-4687-4db7-ad72-c63a2f35dfa2 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.309941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 176/357] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:33.311556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101653f-4687-4db7-ad72-c63a2f35dfa2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.340991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54932594-cf96-4d52-b01f-c95878c7723f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.398111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.398946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.399231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.399506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.463253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9101653f-4687-4db7-ad72-c63a2f35dfa2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.484862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:33.494074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:33.495563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:33.510524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.511351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.511639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.511949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.526699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:33.539921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:33.540829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:33.541596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 177/358] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 231 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:33.542249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.547111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:33.619579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 182/359] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 404 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:33.620387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.669003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-072be3c7-da4a-4000-82d6-94cd18c0f04d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.718190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e0b99d6-a963-4db4-9211-a58127b50a11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.719057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b99d6-a963-4db4-9211-a58127b50a11 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.719344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b99d6-a963-4db4-9211-a58127b50a11 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.719620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e0b99d6-a963-4db4-9211-a58127b50a11 None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.720948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b99d6-a963-4db4-9211-a58127b50a11 None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:33.722392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e0b99d6-a963-4db4-9211-a58127b50a11 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:33.724407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 178/360] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:33.726175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.726467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.726749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.743856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:33.745020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:33.750194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-970118fb-adc9-44b8-931b-529fac30756e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.783294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:33.791638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.792732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.793225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.793704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.805286 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:33.813579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:33.814583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jun 16 11:37:33.815379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:33.817452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 179/362] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:33.817566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 183/362] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/roles/reader => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:33.818181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.818357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:33.829337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:33.936111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7b32ce-c76f-411d-a57b-4534855aeee6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.936949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e2152-7353-4c8b-9fba-be64dce89457 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:33.984729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.985610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.985899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.986179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.988272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:33.989732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:33.990254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:33.990970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:33.991252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:33.991525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:33.994941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:33.994941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:33.995125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 184/363] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:33.995207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:33.995276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.005505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 180/364] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.006950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.057556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.086022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.113669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dd9afd8-d2e8-4c3c-a57a-e3d0d4e31280 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.152202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03bc58ad-0e52-4c91-a4a9-c62414675a9b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.160148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.161051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.161502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.161786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.162693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=85416ef9b7144e4889bdac4457d53735) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:34.163038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.165246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.171242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 185/365] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:33 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c4622473a5824ed7876be10e3da526ca => generated 260 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.172079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.199135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.199842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.200178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.200459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.201242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] RBAC: Adding query filter params (name=reader) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:34.202192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.203733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.216595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 181/366] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.217470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.319308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47e67654-6cf6-4fe6-bd92-6d2d17eff74d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.346922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.355862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df02fa2-6bbb-4e45-9cfe-aa41336bc71d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.363889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.370940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.370940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.370940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.376278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.382939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:34.391043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:34.391976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:34.392759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 186/367] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/roles/admin => generated 83 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:34.393394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.406078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.412897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.412897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.412897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.412897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.412897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.413616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 182/368] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.414363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.535290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c4ea57e-6e72-4ca3-b4cc-b7e76ad0786b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.539511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-055a2e03-92de-45e8-9d3e-8076449668c3 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.580354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.581890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.582393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.582878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.585561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.586439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.586720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.587250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] PATH_INFO: `/v3/users/cdf7f806c890415bbc8f1280abfd838f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.588616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] RBAC: Authorizing `identity:get_user(user_id=cdf7f806c890415bbc8f1280abfd838f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.589912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.596170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 183/369] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/users/cdf7f806c890415bbc8f1280abfd838f => generated 293 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.596831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.643100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.681936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.686869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.688865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.728805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-083058ea-2440-43c4-9aa5-cc0465410816 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.776262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.776980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.777264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.777546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.778299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:34.778610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.780025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.789567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 184/370] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.790365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.819672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 187/371] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:34.820879 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.896553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.903826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddfb4ce-c42f-4149-94a3-938d77c59793 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.926578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:34.931522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd0264a-9cfb-4b91-93c1-a48b4440e1be None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:34.939780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.941547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.941857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.942134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.942988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:34.943285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.945175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:34.959195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 185/372] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 260 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:34.960258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:34.976773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:34.978164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:34.978665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:34.979153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:34.980266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] RBAC: Adding query filter params (role.id=30c1267a61504942b5b1382cc1d47901, user.id=cdf7f806c890415bbc8f1280abfd838f, scope.system=all) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:34.980798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:34.983260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:35.000876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 188/373] 10.222.0.40 () {56 vars in 1308 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/role_assignments?user.id=cdf7f806c890415bbc8f1280abfd838f&scope.system=all&role.id=30c1267a61504942b5b1382cc1d47901 => generated 510 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:35.002058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:35.170384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb7778ad-1033-4533-8a18-3a75811a4462 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:35.254017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d75a67a2-6dff-43ad-9326-8adbdd2fe0fe None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:35.312044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:35.313615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.313615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.315612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:35.357082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:35.372301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:35.373176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.374278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.374566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:35.375815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:35.377348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:35.384028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 189/374] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:35 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 379 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:35.396928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:35.446814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:35.448235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:35.453083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:35.598301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 186/375] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 635 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:35.617903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:35.625904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79fb3171-32ce-4352-a888-57a72f998625 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:35.685596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:35.687498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.688638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.688638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:35.695911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:35.703893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:35.703893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 190/376] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:35 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 307 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:35.715904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:35.753055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:35.816057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9591fbd-1051-467b-914e-21a93e1f27d0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:35.821420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:35.841222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3fa136b-5565-41da-9476-262ca66759d2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:35.878559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:35.879425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.879703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.879996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] PATH_INFO: `/v3/projects/c9c201c143164f7b8064c986ec08f397` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:35.882165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] RBAC: Authorizing `identity:get_project(project_id=c9c201c143164f7b8064c986ec08f397)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:35.883617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:35.886988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 187/377] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:35 2022] GET /identity/v3/projects/c9c201c143164f7b8064c986ec08f397 => generated 344 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:35.891458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:35.920152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:35.921225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.921708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.922180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:35.923262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:35.923785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:35.926531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-863356f0-15c4-4766-b93b-7694c4ca77f0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:35.941670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 191/378] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:35 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c4622473a5824ed7876be10e3da526ca => generated 260 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:35.981585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21722e2-9879-4548-a69c-ce415f45ffd6 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.982613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21722e2-9879-4548-a69c-ce415f45ffd6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.983039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21722e2-9879-4548-a69c-ce415f45ffd6 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:35.984675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 192/379] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:35.990693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:35.991207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:35.991658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.019052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:36.021202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:36.103837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c18cf99f-8f8d-4583-ac57-9b203c193b18 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:36.150483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 193/380] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:36.159239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:36.175094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:36.176215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.179519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.179519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] PATH_INFO: `/v3/groups/79433a1184c146b6949aa9f55f680f94` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.183913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] RBAC: Authorizing `identity:get_group(group_id=79433a1184c146b6949aa9f55f680f94)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:36.185375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:36.187972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:36.199338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 188/381] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:35 2022] GET /identity/v3/groups/79433a1184c146b6949aa9f55f680f94 => generated 229 bytes in 308 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:36.205944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:36.348346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:36.369518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4632d01e-ee64-4a98-9521-1b5eb2aaa08a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:36.394055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37fd451-3f78-4311-a7f8-1204587b0ad1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:36.411332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:36.412302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.412754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.413188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.414605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] RBAC: Adding query filter params (group.id=79433a1184c146b6949aa9f55f680f94, role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=c9c201c143164f7b8064c986ec08f397) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:36.415070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:36.416691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:36.423542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 189/382] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:36 2022] GET /identity/v3/role_assignments?group.id=79433a1184c146b6949aa9f55f680f94&scope.project.id=c9c201c143164f7b8064c986ec08f397&role.id=fda22db004a64c87b35510b321190ad4 => generated 260 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:36.486097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:36.486940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.487234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.487515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.513126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:36.521180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:36.522859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:36.523826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 194/383] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:36 2022] GET /identity/v3/roles/member => generated 84 bytes in 336 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:36.530792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:36.610779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:36.659116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37968432-d530-4ec5-baf6-6365702d47bb None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:36.706487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:36.707402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.707830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.708301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.709234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:36.709686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:36.711210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90be3e37-0eaa-4a3d-9502-50ae304748e9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:36.720779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 190/384] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:36 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:36.726523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:36.835542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1105fe0-b266-4fe1-8cfb-a489169d0cfb None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:36.862842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c9327-c9b2-40dc-b941-bc67e87a9c27 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.863744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c9327-c9b2-40dc-b941-bc67e87a9c27 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.864532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc8c9327-c9b2-40dc-b941-bc67e87a9c27 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.867074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 191/385] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:36 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:36.874434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.875033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.875540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.884817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:36.889992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:36.890491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:36.890995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:36.892483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:36.895515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:36.900461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 195/386] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:36 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:36.901587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:36.903004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:36.905868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:36.944443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:37.054337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 192/387] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:37.079583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4d28203-0fa6-4190-a709-f475796d352f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:37.093516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:37.132665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:37.133877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.134503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.134956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.136706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:37.138323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:37.140989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 196/388] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:36 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:37.151893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:37.192240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:37.283197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6547b9f0-0569-4a52-90ed-705c64154ea5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:37.319190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82da091-8407-4dcd-8c67-c351a8fb369e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:37.321153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:37.355605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:37.356992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.357512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.357628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:37.358134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.358346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.358630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.358905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.361173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] RBAC: Authorizing `identity:create_grant(project_id=85416ef9b7144e4889bdac4457d53735, group_id=fc0b2e61c0d84851839f6e76dcf9ac0a, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:37.362820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:37.377559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:37.389378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:37.390258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:37.391071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:37.393514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 193/389] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:37 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 300 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:37.400528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:37.472717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 197/390] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 11:37:37 2022] PUT /identity/v3/projects/85416ef9b7144e4889bdac4457d53735/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 326 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:37.510510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c17667-8802-45c4-8986-db77a3065e4b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:37.519003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:37.548735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:37.549421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.549759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.550047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.550863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:37.551233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:37.552666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13ed47ea-088a-476c-862b-34058a4b205b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:37.562137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 194/391] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:37 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:37.567715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:37.713394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e28edd0-b5f6-4010-99f3-7e44feda3a51 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:37.722805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01e0f1-e96d-4869-89b2-30d84f0306b7 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.723100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01e0f1-e96d-4869-89b2-30d84f0306b7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.723383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b01e0f1-e96d-4869-89b2-30d84f0306b7 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.730575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 195/392] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:37 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:37.731746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.732109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.732402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.751544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:37.753914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:37.755725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:37.757210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:37.757804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:37.758122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:37.760437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:37.762064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:37.763957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 198/393] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:37 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:37.769569 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:37.783552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:37.841168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 196/394] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:37.863385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:37.963811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8019835c-93c7-4887-ae18-33d2992914c9 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:37.967173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:37.971275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c474a854-37f8-4946-ba6e-8f604e174c5e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.009072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.009886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.010175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.010455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.029232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.030107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.030395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.030762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.032000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:38.033644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:38.035664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 199/395] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:37 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 266 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:38.041454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:38.042976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:38.062695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:38.063620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:38.064450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 197/396] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:38.072860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:38.181852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:38.224430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfea8665-751d-4abc-abf0-42157e01a39b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.267414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-842be1a1-6430-48ca-a704-3636bd81a63f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.301148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.302052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.302329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.302603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.302720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.304054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.304347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.304631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.305392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:38.305711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:38.307096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:38.309150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] RBAC: Authorizing `identity:create_grant(project_id=85416ef9b7144e4889bdac4457d53735, group_id=fc0b2e61c0d84851839f6e76dcf9ac0a, role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:38.310685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:38.316250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 198/397] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:38.322214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:38.345098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:38.402449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 200/398] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 11:37:38 2022] PUT /identity/v3/projects/85416ef9b7144e4889bdac4457d53735/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c4622473a5824ed7876be10e3da526ca => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:38.425981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26b9ef8f-e8df-4856-a6d0-2c10f6cbae95 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.464685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71e9568e-b831-4203-9d84-3fb31aeff86a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.465571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9568e-b831-4203-9d84-3fb31aeff86a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.465860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9568e-b831-4203-9d84-3fb31aeff86a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.466142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71e9568e-b831-4203-9d84-3fb31aeff86a None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.467399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9568e-b831-4203-9d84-3fb31aeff86a None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:38.468900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71e9568e-b831-4203-9d84-3fb31aeff86a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:38.470985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 199/399] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:38 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:38.477085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:38.577675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944807-1d46-4f7d-923c-c1c3efce2819 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.577963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944807-1d46-4f7d-923c-c1c3efce2819 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.578236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab944807-1d46-4f7d-923c-c1c3efce2819 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.584173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 200/400] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:38.585287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.585575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.585887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.607715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:38.609062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:38.626982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61a28026-f0f5-4526-be79-1ef65a84cb72 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.669000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.669978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.670264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.670546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.677097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:38.679234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:38.681384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 201/401] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:38 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:38.691675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:38.693570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 201/402] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:38.712455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:38.718958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:38.752874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:38.875369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0069a7-6ced-4826-9b29-139e3c77b5b8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.919058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850c50ec-4850-4a21-b7fa-7b45f7cb0c98 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:38.926074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a067626-4ce1-4c05-8baa-8ceb86e01cc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.927176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a067626-4ce1-4c05-8baa-8ceb86e01cc5 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.927609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a067626-4ce1-4c05-8baa-8ceb86e01cc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.928100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a067626-4ce1-4c05-8baa-8ceb86e01cc5 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.930296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a067626-4ce1-4c05-8baa-8ceb86e01cc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=90ef71bb81154cd8856ce17bcf6eb121, group_id=fc0b2e61c0d84851839f6e76dcf9ac0a, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:38.931934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a067626-4ce1-4c05-8baa-8ceb86e01cc5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:38.970820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:38.971958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:38.972425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:38.972910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:38.984708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:38.993408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:38.994576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:38.995583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:38.998228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 202/403] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:38 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 280 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:39.004722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.043409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 202/404] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 11:37:38 2022] PUT /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 352 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:39.087345 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d7762-19ce-4e0b-9f74-4fa74cfc8316 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.088135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d7762-19ce-4e0b-9f74-4fa74cfc8316 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.088709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626d7762-19ce-4e0b-9f74-4fa74cfc8316 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.090642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 203/405] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:39 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:39.106266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.106910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.107434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.114688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:39.146196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:39.150146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:39.157599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfc2604b-f80c-425b-9e88-d2ac50165423 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:39.224300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:39.225921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.228135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.228658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.229698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:39.231506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:39.233242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:39.246108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 203/406] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:39 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 242 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:39.251716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.271441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:39.272852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 204/407] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 168 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:39.296638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.376657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-239739bf-0f00-4086-a28f-d7ff6ee63d36 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:39.434370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:39.435243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.435526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.435801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.437849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:39.439298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:39.444210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 204/408] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:39 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:39.450641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.468485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:39.473488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe06e4c0-442a-4d8d-b0be-8da59a88ddf6 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:39.523341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:39.524547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.529120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.529643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.547828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:39.562743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:39.563628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:39.564449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:39.572108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 205/409] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 275 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:39.575537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.657825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c69be891-37b2-476a-8ad4-73d098367c2e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:39.691895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:39.730902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:39.731779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.732089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.732367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.733569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:39.735015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:39.745125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 205/410] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:39 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:39.748901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b5f7c6-376c-4398-83a0-d186bb892aee None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:39.750679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.795347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:39.795978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.796267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.796547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.797317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:39.797628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:39.799004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:39.822485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 206/411] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:39 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:39.828218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:39.882151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:39.902896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8533c213-48db-479c-8d68-8409ae6bd331 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:39.976221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9e390c-6901-4889-81ba-11d0a2058f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:39.977120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e390c-6901-4889-81ba-11d0a2058f18 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:39.977407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e390c-6901-4889-81ba-11d0a2058f18 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:39.977684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9e390c-6901-4889-81ba-11d0a2058f18 None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:39.981228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e390c-6901-4889-81ba-11d0a2058f18 None admin] RBAC: Authorizing `identity:create_grant(project_id=90ef71bb81154cd8856ce17bcf6eb121, group_id=fc0b2e61c0d84851839f6e76dcf9ac0a, role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:39.983126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9e390c-6901-4889-81ba-11d0a2058f18 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:40.013984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64692437-4e1d-4772-bed4-5f6917533ff7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:40.030689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:40.047326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38377afe-2f5b-4cfc-b4bb-9ce2c0a2c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:40.048135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377afe-2f5b-4cfc-b4bb-9ce2c0a2c82a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.048425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377afe-2f5b-4cfc-b4bb-9ce2c0a2c82a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.048705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38377afe-2f5b-4cfc-b4bb-9ce2c0a2c82a None admin] PATH_INFO: `/v3/groups/79433a1184c146b6949aa9f55f680f94` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.049895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377afe-2f5b-4cfc-b4bb-9ce2c0a2c82a None admin] RBAC: Authorizing `identity:get_group(group_id=79433a1184c146b6949aa9f55f680f94)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:40.051337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38377afe-2f5b-4cfc-b4bb-9ce2c0a2c82a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:40.057945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 207/412] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:39 2022] GET /identity/v3/groups/79433a1184c146b6949aa9f55f680f94 => generated 229 bytes in 226 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:40.057945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fcd7a-5be7-463f-8322-f3e7f207481e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.057945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fcd7a-5be7-463f-8322-f3e7f207481e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.057945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0fcd7a-5be7-463f-8322-f3e7f207481e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.057945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 208/413] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:40 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:40.058846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fcd7a-5be7-463f-8322-f3e7f207481e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:40.062505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 206/414] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 11:37:39 2022] PUT /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121/groups/fc0b2e61c0d84851839f6e76dcf9ac0a/roles/c4622473a5824ed7876be10e3da526ca => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:37:40.068539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.069182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.069859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.092781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:40.094455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:40.175388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 207/415] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:40.196843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:40.267682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0fcd7a-5be7-463f-8322-f3e7f207481e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:40.273087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:40.327284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f261e24-ac56-43da-91eb-cac5c8ac4321 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:40.335241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:40.336164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.336454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.336735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] PATH_INFO: `/v3/projects/c9c201c143164f7b8064c986ec08f397` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.338975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] RBAC: Authorizing `identity:get_project(project_id=c9c201c143164f7b8064c986ec08f397)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:40.340491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:40.343503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 209/416] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:40 2022] GET /identity/v3/projects/c9c201c143164f7b8064c986ec08f397 => generated 344 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:40.350490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:40.383951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:40.384865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.385214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.385552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] PATH_INFO: `/v3/roles/member` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.399995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:40.409476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:40.410575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:40.411536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:40.414420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 208/417] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:40 2022] GET /identity/v3/roles/member => generated 84 bytes in 218 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:40.419593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:40.448974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:40.570090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23c0c6f6-9a2f-4b85-a814-d8f7b3a1a877 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:40.572699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85c86de8-12e7-4668-ab84-79a9b835b8d8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:40.602987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:40.616611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:40.620671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.621137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.628049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.628049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] RBAC: Adding query filter params (name=member) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:40.628049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:40.628049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:40.640921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:40.641976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.642371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.642753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] PATH_INFO: `/v3/projects/c9c201c143164f7b8064c986ec08f397/groups/79433a1184c146b6949aa9f55f680f94/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.644909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] RBAC: Authorizing `identity:create_grant(project_id=c9c201c143164f7b8064c986ec08f397, group_id=79433a1184c146b6949aa9f55f680f94, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:40.646094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 209/418] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:40 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:40.648064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:40.656891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:40.722621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 210/419] 10.222.0.40 () {58 vars in 1442 bytes} [Thu Jun 16 11:37:40 2022] PUT /identity/v3/projects/c9c201c143164f7b8064c986ec08f397/groups/79433a1184c146b6949aa9f55f680f94/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 373 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:37:40.787077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d42098-9b94-4e1d-8b15-4f7e743b9ef5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:40.825609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ce796a7-856e-4444-aa27-6e5a3d84d89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:40.826471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce796a7-856e-4444-aa27-6e5a3d84d89c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:40.826856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce796a7-856e-4444-aa27-6e5a3d84d89c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:40.827240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ce796a7-856e-4444-aa27-6e5a3d84d89c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:40.890333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:40.904075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce796a7-856e-4444-aa27-6e5a3d84d89c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:40.908584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ce796a7-856e-4444-aa27-6e5a3d84d89c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:41.017473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 210/420] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 361 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:41.023086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.074259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32906a5-6853-4147-8f29-2061776e0b37 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.074558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32906a5-6853-4147-8f29-2061776e0b37 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.074824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e32906a5-6853-4147-8f29-2061776e0b37 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.076532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 211/421] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:41.081894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.082191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.082479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.101160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:41.102880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:41.136415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd58a8e-265c-49ef-8ad9-a4503d63627f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:41.151306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:41.184209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 212/422] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:41.192570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:41.194100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.194389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.194669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.195914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:41.197365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:41.199248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 211/423] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:41.204247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.204654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.228292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:41.315904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5d53977-c2d2-42bc-b758-800f54fceb57 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:41.363590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:41.364793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.365809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.366325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.386054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:41.388825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-641227ec-b5a0-4752-84a4-da66a8184738 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:41.397889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:41.398844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:41.399624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:41.410941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 213/424] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:41.418165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.450873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:41.452241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.452843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.453323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.455049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:41.457153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:41.460006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 212/425] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:41.469016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.559033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:41.576221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:41.614664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bfad830-20d3-45c3-ac35-d5b8142f3bde None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:41.641950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b53e684-8e98-420d-9560-d0f8936f2e0e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:41.660861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:41.661613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.661836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.662646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.663503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=85416ef9b7144e4889bdac4457d53735) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:41.663817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:41.665295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff65a188-884f-45da-8a68-45bc1193092d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:41.692046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:41.692204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 213/426] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 611 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:41.692724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.693006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.693322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.694079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:41.694388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:41.695769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:41.709916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847af5e-f594-46f5-8603-f8b96275b337 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.710213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847af5e-f594-46f5-8603-f8b96275b337 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.710480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e847af5e-f594-46f5-8603-f8b96275b337 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.716135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 214/427] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:41 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:41.717754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9089-7474-41d5-b722-406e4384aecc None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.717938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 214/428] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 300 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:41.718268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9089-7474-41d5-b722-406e4384aecc None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.718648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094e9089-7474-41d5-b722-406e4384aecc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.724041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.759898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-094e9089-7474-41d5-b722-406e4384aecc None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:41.761116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-094e9089-7474-41d5-b722-406e4384aecc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:41.840571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-561c4a8e-7c49-4626-9606-922e950acfd6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:41.908074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:41.908739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:41.909020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:41.909296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:41.928119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:41.942337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 215/429] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 225 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:41.967730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e9089-7474-41d5-b722-406e4384aecc None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:41.991749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:41.993735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.031297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.096796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094e9089-7474-41d5-b722-406e4384aecc None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.132901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 215/430] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 409 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:42.140468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.143390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.144226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.144508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.144781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] PATH_INFO: `/v3/roles/member` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.160285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:42.171719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:42.172669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jun 16 11:37:42.173439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:42.175000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 216/431] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:41 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:42.182183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.203714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.281832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e37081e9-9f40-41a8-ad09-6bd3ffe4e0fb None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.302052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65fe5c03-f096-48cf-bfb0-5c04d13026cc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.305977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.325937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.327510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.328208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.328630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] PATH_INFO: `/v3/projects/85416ef9b7144e4889bdac4457d53735` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.330160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] RBAC: Authorizing `identity:get_project(project_id=85416ef9b7144e4889bdac4457d53735)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:42.332409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.335359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 216/432] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/projects/85416ef9b7144e4889bdac4457d53735 => generated 298 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:42.341163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.344006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.345021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.345447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.346728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.347633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] RBAC: Adding query filter params (name=member) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:42.348120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:42.350169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.360707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 217/433] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:42.366603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.485147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188a53b0-4300-4a86-a0dc-80be1bef211a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.497434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.503163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e6dd44-202d-46b0-8dbc-b1bf11badd60 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.525793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8415fe7d-3095-49d6-ba37-ea3472fd7ffe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.526893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415fe7d-3095-49d6-ba37-ea3472fd7ffe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.527403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415fe7d-3095-49d6-ba37-ea3472fd7ffe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.527823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8415fe7d-3095-49d6-ba37-ea3472fd7ffe None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.529169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415fe7d-3095-49d6-ba37-ea3472fd7ffe None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:42.530749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8415fe7d-3095-49d6-ba37-ea3472fd7ffe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.532831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 217/434] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:42.538682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759f15f-53fe-4cb5-8dc1-04101ce783cb None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.538970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759f15f-53fe-4cb5-8dc1-04101ce783cb None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.539260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8759f15f-53fe-4cb5-8dc1-04101ce783cb None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.541741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 218/435] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:42.542614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759f15f-53fe-4cb5-8dc1-04101ce783cb None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.543826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.544411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.544732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.545021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.590004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.633145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:42.634589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.666824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8759f15f-53fe-4cb5-8dc1-04101ce783cb None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.707997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.709518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.709518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.709764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.710531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=85416ef9b7144e4889bdac4457d53735) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:42.710843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:42.712609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac861afa-f7da-454c-beec-db59dd20adf3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.728039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 219/436] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=85416ef9b7144e4889bdac4457d53735&role.id=c4622473a5824ed7876be10e3da526ca => generated 611 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:42.728039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.728039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.728039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.744548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:42.745697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:42.756155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 218/437] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 390 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:42.761924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.796149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.834663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 220/438] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:42 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:42.838078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:42.863499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.884976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f2130b-f573-45d4-bf5d-0c2a49924bb8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:42.939369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:42.940928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:42.940928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:42.940928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:42.945493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:42.947001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:42.951939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 219/439] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:42.954479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:42.992324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea9b6bae-a828-470f-8000-2a2fab41857b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.030059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.030805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.031088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.031364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.046713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:43.057432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:43.058364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:43.059137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 221/440] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:43.065047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.092089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3888837-fcc8-48fe-88fc-26b82a4d4d5d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.109951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.139192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.140132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.140505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.140809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.142012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.143464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.145731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 220/441] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:42 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:43.151933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.155337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.214463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d284cf78-0741-4982-bedd-380d32330198 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.254843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.255517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.256325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.256608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.257366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:43.257680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.259058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12d0c0e0-2219-4432-ae0b-7518212769c3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.266945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 222/442] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:43.268176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c973786-4082-4516-bcab-841470ef1513 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.268463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c973786-4082-4516-bcab-841470ef1513 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.268740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c973786-4082-4516-bcab-841470ef1513 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.269958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 223/443] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:43.272913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c973786-4082-4516-bcab-841470ef1513 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.284043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aab6c54e-5f0f-4cfe-aa6c-62d395556d01 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.315928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.316581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.316867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.317145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.317972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c2d46bdeda48498b8c04ea76027be796, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:43.318807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.320251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84ff9600-017e-489e-b582-4fa3945feb82 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.325921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 221/444] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c2d46bdeda48498b8c04ea76027be796 => generated 611 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:43.327073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.327361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.327645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.350420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:43.351825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:43.369079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.382447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c973786-4082-4516-bcab-841470ef1513 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.420001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.421527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.422200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.422491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.422770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.428464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 222/445] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:43.452871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.502756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.504924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.565826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d64c95c-1398-49cc-9be0-ee2f6ebe5c38 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.607758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.609325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.609325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.609325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.611014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.621775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 224/446] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:43.622034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:43.627365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:43.631462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:43.632794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:43.633786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 223/447] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:43.639703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.713497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.746973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e8f972-2ff7-408e-8289-b06b33114841 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.756518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-794ee3ce-c827-41b8-99f3-f1c49243b1e8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.779789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.780674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.780957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.781240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] PATH_INFO: `/v3/projects/90ef71bb81154cd8856ce17bcf6eb121` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.782421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] RBAC: Authorizing `identity:get_project(project_id=90ef71bb81154cd8856ce17bcf6eb121)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.783888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.785569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 225/448] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/projects/90ef71bb81154cd8856ce17bcf6eb121 => generated 302 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:43.789088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.789754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.790038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.790313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.791057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:43.791363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.791448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.792757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.800787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 224/449] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:43.806611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.849108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.888613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e947d170-fcad-445e-9b5f-938be27339bf None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.902101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2aca1f5-0cca-4736-9319-0b84b89a3539 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:43.919007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:43.921515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.922370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.922653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.922929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] PATH_INFO: `/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:43.924135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] RBAC: Authorizing `identity:get_group(group_id=fc0b2e61c0d84851839f6e76dcf9ac0a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:43.925588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:43.927191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 226/450] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/groups/fc0b2e61c0d84851839f6e76dcf9ac0a => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:43.933347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:43.935819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:43.936500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:43.936790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:43.937071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:44.002679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:44.004153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:44.029874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9499992-a52e-4114-96a2-0738dad926b7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:44.056473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:44.064402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:44.065043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:44.065329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:44.069920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:44.069920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] RBAC: Adding query filter params (group.id=fc0b2e61c0d84851839f6e76dcf9ac0a, role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=90ef71bb81154cd8856ce17bcf6eb121) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:44.069920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:44.069920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:44.073859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 227/451] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/role_assignments?group.id=fc0b2e61c0d84851839f6e76dcf9ac0a&scope.project.id=90ef71bb81154cd8856ce17bcf6eb121&role.id=c4622473a5824ed7876be10e3da526ca => generated 611 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:44.102311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 225/452] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 358 bytes (2 switches on core 0) Jun 16 11:37:44.107900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:44.213545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92ca186c-4153-4518-a399-4f08c0f7c246 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:44.248308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:44.249174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:44.249463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:44.249750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] PATH_INFO: `/v3/projects/c9c201c143164f7b8064c986ec08f397` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:44.250981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] RBAC: Authorizing `identity:get_project(project_id=c9c201c143164f7b8064c986ec08f397)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:44.252456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:44.254162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 228/453] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:37:44 2022] GET /identity/v3/projects/c9c201c143164f7b8064c986ec08f397 => generated 344 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:44.259797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:44.278901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:44.360010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-225616e0-1379-4b7d-830d-f6cd019502d5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:44.392906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fceee99b-3b03-4622-9d0d-af9ab3b8a473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:44.393767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceee99b-3b03-4622-9d0d-af9ab3b8a473 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:44.394052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceee99b-3b03-4622-9d0d-af9ab3b8a473 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:44.394327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fceee99b-3b03-4622-9d0d-af9ab3b8a473 None admin] PATH_INFO: `/v3/groups/79433a1184c146b6949aa9f55f680f94` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:44.395490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceee99b-3b03-4622-9d0d-af9ab3b8a473 None admin] RBAC: Authorizing `identity:get_group(group_id=79433a1184c146b6949aa9f55f680f94)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:44.397491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fceee99b-3b03-4622-9d0d-af9ab3b8a473 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:44.399114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 226/454] 10.222.0.40 () {56 vars in 1180 bytes} [Thu Jun 16 11:37:44 2022] GET /identity/v3/groups/79433a1184c146b6949aa9f55f680f94 => generated 229 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:44.405145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:44.463563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:44.502644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e687e198-f109-4b5c-abd5-03462c778347 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:44.534983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:44.535623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:44.535930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:44.536208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:44.537067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] RBAC: Adding query filter params (group.id=79433a1184c146b6949aa9f55f680f94, role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=c9c201c143164f7b8064c986ec08f397) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:44.537380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:44.538794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d93df44a-a012-4b3a-a14c-5b2035a70e7f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:44.544482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 229/455] 10.222.0.40 () {56 vars in 1376 bytes} [Thu Jun 16 11:37:44 2022] GET /identity/v3/role_assignments?group.id=79433a1184c146b6949aa9f55f680f94&scope.project.id=c9c201c143164f7b8064c986ec08f397&role.id=fda22db004a64c87b35510b321190ad4 => generated 611 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:47.191737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb005f-e3e9-4a07-b270-c60e9e9d9834 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.192273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb005f-e3e9-4a07-b270-c60e9e9d9834 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.192382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9eb005f-e3e9-4a07-b270-c60e9e9d9834 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.194435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 227/456] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:47.199334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697b070-b7bc-49b4-9eee-2fe64efe20a6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.199749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697b070-b7bc-49b4-9eee-2fe64efe20a6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.199945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f697b070-b7bc-49b4-9eee-2fe64efe20a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.221176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f697b070-b7bc-49b4-9eee-2fe64efe20a6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:47.222370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f697b070-b7bc-49b4-9eee-2fe64efe20a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:47.309460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcf360-f0c5-411d-a868-fd608e1dd4f1 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.310143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcf360-f0c5-411d-a868-fd608e1dd4f1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.310642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1bcf360-f0c5-411d-a868-fd608e1dd4f1 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.313630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 230/458] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:47.313858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 228/458] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:47.319891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.319891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.319891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.340467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcf360-f0c5-411d-a868-fd608e1dd4f1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:47.344907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:47.346086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:47.445016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:47.468885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 231/459] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:47.497475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:47.508219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:47.529488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1bcf360-f0c5-411d-a868-fd608e1dd4f1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:47.573268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:47.574772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.575216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.575635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.589952 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:47.592382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:47.599624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87509a8-e6b3-4391-908c-a72825f77858 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-f3afe3dc-2110-4def-82d2-726412747b30 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:47.614687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:47.615827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:47.616840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:47.616840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:47.616840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:47.616840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:47.616840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 229/460] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:47.618001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065f8a-6e53-46cc-8622-8fe89d7843fb None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.618293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065f8a-6e53-46cc-8622-8fe89d7843fb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.618574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51065f8a-6e53-46cc-8622-8fe89d7843fb None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.620438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 230/461] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:47.624906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065f8a-6e53-46cc-8622-8fe89d7843fb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:47.647026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:47.647826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.648136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.648448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.659958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:47.661416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-73262390-3e60-4cf2-b867-a90b18c5a2c3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:47.681921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:47.682819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:47.683664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:47.683664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:47.683664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:47.683664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:47.688032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 232/462] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:47.690051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.690051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.690285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.710866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:47.712481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:47.746716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:47.779555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51065f8a-6e53-46cc-8622-8fe89d7843fb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:47.788261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 233/463] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:47.789193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:47.825900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:47.827471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.827926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.828348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.829213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:47.829666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:47.831189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdf7e6e8-7909-4b21-bc9e-505dd50d042a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:47.838767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:47.845305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 231/464] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:47.846749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4dc7-2945-4745-a00b-80c77cbe1c71 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.847208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4dc7-2945-4745-a00b-80c77cbe1c71 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.847631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-773a4dc7-2945-4745-a00b-80c77cbe1c71 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.851834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 232/465] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:47.851834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a4dc7-2945-4745-a00b-80c77cbe1c71 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:47.902333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f4b0f6a-8f83-4415-b7b9-3b09749dd051 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:47.945080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:47.946040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:47.946332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:47.946615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:47.947344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:47.947661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:47.949230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:47.961150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 234/466] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:47.961716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:47.963352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-773a4dc7-2945-4745-a00b-80c77cbe1c71 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:47.996164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:48.003309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.005560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.005560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.005560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.016011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.017464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-23900dca-29a5-4f29-a6f6-e993c3fabefa None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:48.034793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:48.035820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:48.036766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:48.036766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:48.036766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:48.036766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:48.036766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 233/467] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:48.037294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.037577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.037848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.055723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:48.056950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:48.078486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24b51f94-f913-4bb1-a2e3-d2be4723c3c8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:48.108102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:48.119653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.120597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.120879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.121147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.121851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.123290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.129054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 234/468] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:48.130082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.231371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bf4f996-e0c7-48b7-8a61-a1a74fa83034 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:48.231991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 235/469] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:47 2022] POST /identity/v3/users => generated 248 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:48.232353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.280018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.281024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.281483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.281926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.282864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.284730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.332934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:48.352751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-716e6791-a437-4eec-8289-a1c1f968c322 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:48.381055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 235/470] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:48 2022] POST /identity/v3/users => generated 246 bytes in 250 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:48.382124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.429423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.430117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.430407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.430688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.431469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:48.431785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.434511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.452598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 236/471] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:48.461016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.474000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:48.560714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac42289-7e3b-4d4e-a77d-07801fb8b998 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:48.586244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-073e2f95-b683-4bef-af8b-69af7cd65f12 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:48.604716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.605745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.606168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.606607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.624797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.626322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.626685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a30bfad-6ec9-4282-9cc1-8948479af625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.627372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30bfad-6ec9-4282-9cc1-8948479af625 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.627710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30bfad-6ec9-4282-9cc1-8948479af625 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.628025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a30bfad-6ec9-4282-9cc1-8948479af625 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.628744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30bfad-6ec9-4282-9cc1-8948479af625 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.630191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a30bfad-6ec9-4282-9cc1-8948479af625 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-120d2be9-6214-4325-92e8-11e7fff3940c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:48.644398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:48.645334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:48.647175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:48.647175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:48.647175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:48.647175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:48.647175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 236/472] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:48 2022] GET /identity/v3/domains/Default => generated 87 bytes in 264 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:48.647743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85b5c-1f3f-4eaa-9b5c-2c140e7777a1 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.647743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85b5c-1f3f-4eaa-9b5c-2c140e7777a1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.648048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aa85b5c-1f3f-4eaa-9b5c-2c140e7777a1 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.649178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 237/473] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:48.653008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa85b5c-1f3f-4eaa-9b5c-2c140e7777a1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.729364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:48.737829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 237/474] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:48 2022] POST /identity/v3/users => generated 248 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:48.739314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.739609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.739937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.758310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:48.759190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:48.760434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:48.781731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aa85b5c-1f3f-4eaa-9b5c-2c140e7777a1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:48.828584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:48.829380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:48.829672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:48.830328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:48.831461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:48.831784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:48.834532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:48.844022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 238/475] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:48 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:48.847375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 238/476] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:48 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:48.854803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.865921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:48.976622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def34211-5344-400b-a35d-fb6a89dba1b9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:49.000342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a18a27c-8afc-4e80-a014-d1e918a029dd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:49.016253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:49.034250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:49.035036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:49.035330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:49.035609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:49.037068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:49.037960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:49.043266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:49.043973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:49.044266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:49.044552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:49.045319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] RBAC: Authorizing `identity:create_role()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:49.046923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:49.122669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.sql.core [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 11:37:49.122669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 11:37:49.122669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [parameters: {'id': '69ca842b38ea414fb6644ddf59db5c09', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 11:37:49.122669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87092) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 11:37:49.130345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 239/477] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:48 2022] POST /identity/v3/users => generated 249 bytes in 277 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application conn.query(q) Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result.read() Jun 16 11:37:49.143761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return ref.to_dict() Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 11:37:49.144764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application next(self.gen) Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application yield resource Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application next(self.gen) Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise self.value Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application session.commit() Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 11:37:49.146738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.session.flush() Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application compat.raise_( Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 11:37:49.147618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 11:37:49.149644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application conn.query(q) Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result.read() Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 11:37:49.152793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [parameters: {'id': '69ca842b38ea414fb6644ddf59db5c09', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:49.154759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jun 16 11:37:49.159284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:49.161451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 239/478] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 11:37:48 2022] POST /identity/v3/roles => generated 144 bytes in 280 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:49.164360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:49.323953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-354f11ce-69c2-4ac0-95b2-87dcdf532d1b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:49.369164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53731963-54d9-4e5d-a954-09e0d43f15c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:49.369989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53731963-54d9-4e5d-a954-09e0d43f15c8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:49.370276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53731963-54d9-4e5d-a954-09e0d43f15c8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:49.370556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53731963-54d9-4e5d-a954-09e0d43f15c8 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:49.384417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53731963-54d9-4e5d-a954-09e0d43f15c8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-53731963-54d9-4e5d-a954-09e0d43f15c8 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:49.392630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:49.393510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:49.394708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 240/479] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:37:49 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:49.401240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:49.422306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:49.529782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060c76c6-1a53-4b1a-b72c-8e0ba9cfedfa None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:49.573877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:49.574561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:49.574846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:49.575123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:49.575886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:49.576214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:49.577575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32ef140c-f6f2-4f4c-a67c-568e6e1005a5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:49.588251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 240/480] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:49 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:50.921350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce684e4c-2fe4-4818-9e9d-96526996f540 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:50.921851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce684e4c-2fe4-4818-9e9d-96526996f540 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:50.921924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce684e4c-2fe4-4818-9e9d-96526996f540 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:50.923477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 241/481] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:50.925766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddb3c-fa8e-4a58-8471-dff388078d26 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:50.926148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddb3c-fa8e-4a58-8471-dff388078d26 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:50.926334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dcddb3c-fa8e-4a58-8471-dff388078d26 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:50.927955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 241/482] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:50.932810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:50.933097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:50.933379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:50.939521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:50.939731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:50.940036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:50.956429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:50.957617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:50.976746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:50.977924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:51.035396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.086429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 242/483] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 154 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:51.106425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 242/484] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:51.119253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.142307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.145300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.295193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea59379b-bd2d-4b9a-be95-80031bc675e4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.303695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc0c600-7fae-4435-9764-7032d91aa48f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.334794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53c56b-0b38-410b-ad15-4a1383911105 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.335853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c56b-0b38-410b-ad15-4a1383911105 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.336197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c56b-0b38-410b-ad15-4a1383911105 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.336856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53c56b-0b38-410b-ad15-4a1383911105 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.341793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d67704-3894-43cb-b3a2-7fa79938ea78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.342686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67704-3894-43cb-b3a2-7fa79938ea78 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.343035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67704-3894-43cb-b3a2-7fa79938ea78 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.343376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d67704-3894-43cb-b3a2-7fa79938ea78 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.349266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53c56b-0b38-410b-ad15-4a1383911105 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:51.354260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d67704-3894-43cb-b3a2-7fa79938ea78 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-fb53c56b-0b38-410b-ad15-4a1383911105 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:51.358241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:51.359112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:51.359927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 243/485] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:51.361078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaec97-9c96-4509-8487-52abc6288040 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.363908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaec97-9c96-4509-8487-52abc6288040 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.363908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8aaec97-9c96-4509-8487-52abc6288040 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.363908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 244/486] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-87d67704-3894-43cb-b3a2-7fa79938ea78 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:51.366882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:51.368751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:51.370565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:51.376295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaec97-9c96-4509-8487-52abc6288040 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.376457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 243/487] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:51.376457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.376457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.376879 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.395488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:51.397437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:51.413742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.479207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.490923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 244/488] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:51 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:37:51.492611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.495198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8aaec97-9c96-4509-8487-52abc6288040 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.532753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.533520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.533903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.534281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.535193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:51.535702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:51.537424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:51.547231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 245/489] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:51.552095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.616725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1fce84-b7c6-400c-9980-041b10e4cbd3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.652016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.653045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.653532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.654033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.654978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:51.655505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:51.657113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:51.658607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.665336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cdd9e3-43ac-4227-8965-d85d552287cf None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.671161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 245/490] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:51.671161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.702475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.703280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.703564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.703840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.715061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:51.723158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:51.724686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:51.726418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 246/491] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/roles/service => generated 85 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:51.727829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.728662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.780588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07f9ee01-e96d-40d4-a60e-f54e31db28dc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.816608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.817432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.817728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.818561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.830510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5306f0f3-c65c-468c-b6f2-4978434e806a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.831019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:51.832877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:51.851354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:51.853121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:51.854901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:51.854901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:51.854901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:51.854901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:51.854901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 246/492] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:51.854901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.867415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:51.868347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:51.868702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:51.869046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:51.880905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:51.882484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:51.899948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:51.901697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:51.903437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:51.903437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:51.903437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:51.903437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:51.903437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 247/493] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:51.904064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:51.914921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:51.971567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0073b45f-9753-4cb6-80fc-c0ddef2cca10 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:51.983796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.011907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.012604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.012820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e7f520-4e3e-4ba7-91aa-193021c5f562 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.013298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.013298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.013936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:52.014248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.016568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.033665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 247/494] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:52.033665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.050398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.051236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.051526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.051802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.053182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:52.053490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.054904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8509bb5-a270-4132-9a5d-a075a5640277 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.067379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 248/495] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:52.068700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f6d45-9c50-42cb-9fdb-dd385978bcf4 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.068984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f6d45-9c50-42cb-9fdb-dd385978bcf4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.069256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-839f6d45-9c50-42cb-9fdb-dd385978bcf4 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.071913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 249/496] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:52.071913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f6d45-9c50-42cb-9fdb-dd385978bcf4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.144092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.150463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42f0804c-6984-4447-9f70-36eac4faf441 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.185344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-839f6d45-9c50-42cb-9fdb-dd385978bcf4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.187967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.188704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.189021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.189360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.190079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:52.190388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.191750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.204124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 248/497] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:52.204956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.223249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.224635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.224920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.225202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.235756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.237227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.241483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-2a4755c9-23dc-4760-addf-11b8b1f7acf7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:52.256039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:52.256904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:52.257784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:52.257784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:52.257784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:52.257784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:52.257784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 250/498] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:52.258480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.258764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.259021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.275215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:52.276454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:52.319650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb9bd40c-d6c3-47f0-8ef7-4fba1f6e57b7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.347922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 251/499] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:52.347922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.366299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.367098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.367382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.367654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.378019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.380135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:37:52.396966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:52.398031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:37:52.399119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:52.402756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 249/500] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:52.402917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.403484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.461219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f938baa8-3346-4118-a29b-a9b3d50fc31a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.496786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.497587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.497872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.498149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] PATH_INFO: `/v3/users/glance` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.502702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.505077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.506561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.512067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744a4471-df52-46a2-b625-a4fa38c2d32c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:37:52.522850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:52.523714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:52.524611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 252/501] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:52.525880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.549462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.550124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.550411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.550693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.551414 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:52.551723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.553147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.564014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 250/502] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:52.564793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.633248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56daf234-1a89-4b55-873e-652872f7adde None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.643660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.663727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbf6ccc9-e499-4730-b354-ca988bce17fb None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.667624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f853842-e656-4954-a9b5-2188f51a1c93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.668445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853842-e656-4954-a9b5-2188f51a1c93 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.668726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853842-e656-4954-a9b5-2188f51a1c93 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.668997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f853842-e656-4954-a9b5-2188f51a1c93 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.680098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f853842-e656-4954-a9b5-2188f51a1c93 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-2f853842-e656-4954-a9b5-2188f51a1c93 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:52.687891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:52.688963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:52.689801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 253/503] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:52.690446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7d84-1fd9-45ef-9507-9dad790b8b72 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.690628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7d84-1fd9-45ef-9507-9dad790b8b72 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.690910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83bb7d84-1fd9-45ef-9507-9dad790b8b72 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.696283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 254/504] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:52 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:52.696283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7d84-1fd9-45ef-9507-9dad790b8b72 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.703476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.704213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.704506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.704786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.705631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:52.705946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.707384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.745014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 251/505] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:52.745828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.756813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.797717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83bb7d84-1fd9-45ef-9507-9dad790b8b72 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.829868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.830539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.830824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.831102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.831966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:52.832283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.834278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.844771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3c475c-a02d-430d-b054-7aeba98b2c23 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.869473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 255/506] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:52.870418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:52.883101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:52.883991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.884225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.884500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.891689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:52.893184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:52.898982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-94dbcf6f-3533-4191-a767-51a0a6c7595c None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:37:52.908167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:52.909205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:52.910094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 252/507] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/users/cinder => generated 84 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:52.910876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:52.911125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:52.911669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:52.927051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:52.928284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:52.972552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048aef4a-70e7-4ec3-8dcd-c0f8f64c4b17 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:52.988732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:52.993929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 253/508] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:52.994723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.006054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.006733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.007279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.007566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.008351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:53.008667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.010129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.018229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 256/509] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:53.019031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.103977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bee43d1c-6332-49f8-a528-847c34ef110e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.107519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.117116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-420f3b88-913f-4fb4-976e-b125dcb5807a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.145568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.145568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.145568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.145568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.149791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61572d8-b127-4666-adff-3720407906dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.150429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61572d8-b127-4666-adff-3720407906dd None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.150714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61572d8-b127-4666-adff-3720407906dd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.150990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a61572d8-b127-4666-adff-3720407906dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.201954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.205402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.206850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.216981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61572d8-b127-4666-adff-3720407906dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.218435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a61572d8-b127-4666-adff-3720407906dd None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.303953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 254/510] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 309 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:53.304121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.304774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.318889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 257/511] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:53.319690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61572d8-b127-4666-adff-3720407906dd None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.397797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.401200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f93db5-fdc1-4fd4-af6f-13472cdea406 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.415186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a61572d8-b127-4666-adff-3720407906dd None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.433906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.434757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.435389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.435677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.436548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:53.436865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.438309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.450004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.450788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.451073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.451352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.455763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 255/512] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:53.456581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.465382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.466847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:53.484897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:53.485764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:53.486595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:53.486595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:53.486595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.486595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:53.486595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 258/513] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:53.486861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.529028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.557087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8ba6b85-d789-4860-ac54-ccb8c47440eb None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.582550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569032bb-c363-4cf0-9ee4-2571ee04887d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.589929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.590716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.591006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.591287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.601576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.603016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.616455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.617231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.617512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.617784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:53.620801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:53.621753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:53.622753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:53.622753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:53.622753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.622753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:53.622753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 256/514] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:53.625114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.627671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.629142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.636325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:53.646761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:53.649760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:53.650577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:53.650577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:53.650577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.650577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:53.650577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 259/515] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:53.650577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.729142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c0cf9a-4cc8-4240-b208-2067a9ee0913 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.743224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.749661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe4a0c43-cfe7-435b-b8d7-b663b25d6767 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.762263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.763059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.763340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.763616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.774323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.775777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.782085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.782729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.783017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.783298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:53.792717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:53.793538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:53.794847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:53.794847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:53.794847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:53.794847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:53.794847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 257/516] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:53.794847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.852357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.852527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.853817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.892237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be209e54-2741-4b44-86cf-3e6c4e43f79a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:53.925739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:53.926329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:53.926616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:53.926900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:53.927744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:53.928087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:53.929462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:53.941661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 258/517] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:53.942447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:53.942807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:53.950942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 260/518] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 302 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:53.951990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.039410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537740d6-7e00-484a-976a-23956675cdd4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.052322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d78a02-3681-4f71-8d38-69d4f32aefe7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.058743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.072116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.072772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.073055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.073331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.074041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:54.074346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.075698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.085232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.085887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.086171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.086528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.087645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:54.087920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.088064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 259/519] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:54.089114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.089376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.101804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 261/520] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:54.102500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.156495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.190687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be94ec8-6a5f-4002-9bea-ecd6eb0865fe None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.199886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683e15ba-0e2f-49a1-a3e8-988f14b53968 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.225720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.226391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.226679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.226958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.227677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:54.228021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.229389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.233469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.234131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.234414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.234689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.235388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.236832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.240741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 260/521] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:54.240741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.283593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.333005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 262/522] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:37:54 2022] POST /identity/v3/users => generated 247 bytes in 231 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:37:54.334367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.339086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94fc339d-6cf5-4a10-ab8c-bbb7f96584e9 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.385173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.386008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.386303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.386593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.399280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.400926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:54.421486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:54.422562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:54.423403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:54.423403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:54.423403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:54.423403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:54.423403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 261/523] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:54.423663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.468317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e274b54c-0836-4103-9b7a-65297f2be326 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.479937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.506713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.507527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.507815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.508793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.517153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.518612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.530336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:37:54.535152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:54.536297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 263/524] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/users/neutron => generated 85 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:54.538335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.542999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b674e9db-1d68-4f82-b31b-7acc8a33cc9f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.581619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.582527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.582876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.583216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.591124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.592695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:37:54.607006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:54.607843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:54.608717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 262/525] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:54.609276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.655824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da42119-6a00-4e9b-8076-7738b0aa716b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.698700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.699594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.700024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.700405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.708223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.709797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.722366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8b3eb0-40df-464c-8ce2-3eb1f677e72a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:37:54.724900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:54.725732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:54.726647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 264/526] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:54.727204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.761527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.762179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.763259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.763704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.764014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.764735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:54.765050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.766436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.782713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 263/527] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:54.783413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.797618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:54.835486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d2728f-a8f0-40cd-af12-b66df50830ba None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.870595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.871273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.871559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.871836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.872703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:54.873014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.874480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.889774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38b36373-c495-457f-8d5c-a0bd195fe422 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:54.892774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 265/528] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:54.893613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.935771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:54.936780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:54.937224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:54.937622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:54.938415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:54.938888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:54.940977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:54.959801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 264/529] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:54.960654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:54.987977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.000071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a9b210-aa52-43c2-a846-1d0cfbcbbefe None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.035396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.038557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.039243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.039531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.039835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.040646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:55.040943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.042388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.060361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 266/530] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:55.061175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.070598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e79f0-190e-40a5-b293-f8e13421aff7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.105933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.106731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.107013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.107292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.114398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.116585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:37:55.130903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:55.131842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:55.132727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 265/531] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:54 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:55.133208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.169002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b8de52-81d6-4449-a79d-5e7e6ecb8209 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.202564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.203217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.203502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.203780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.221373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.238203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc17f02a-1bbe-4f30-a562-dbf638c56af2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.271109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.271468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.273064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.276217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.276940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.277231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.277513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.278396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=54986964e91b4af7997e09d4682cc31e) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:55.278721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.280400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.296104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 266/532] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/role_assignments?user.id=54986964e91b4af7997e09d4682cc31e&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:55.296104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.409024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 267/533] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:55.409892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.438626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1944bcc4-96ca-4fd2-9a6f-e41586ff22c1 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.477373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.493509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.494424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.494730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.495015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.496187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=35d68b3bee5445849c59da135df06fea) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:55.496509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.498025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.514690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 267/534] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/role_assignments?user.id=35d68b3bee5445849c59da135df06fea&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:55.514690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.573124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e564c42d-0f36-4a0b-b307-91ff4599cc09 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.647922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.647922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.647922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.647922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.648342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:55.648342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.649631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.664107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 268/535] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:55.669256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.670513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9aaad4-0356-4738-a877-9f6d0fa43967 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.715645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.716625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.717014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.717390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.729540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.731235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:55.749988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:55.750956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:55.751795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:55.751795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:55.751795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:55.751795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:55.751795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 268/536] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 240 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:55.757802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:55.792979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d37d764a-319f-496f-9928-fa4aa1d02aef None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.802950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.830421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.831392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.831901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.832366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.833970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=72c24cb67566467dbdf65cfd44a59274) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:55.834461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.836314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.843523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 269/537] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/role_assignments?user.id=72c24cb67566467dbdf65cfd44a59274&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:55.869172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3551c6dd-7647-4a7d-9bba-001498362e3a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:55.885580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:55.908887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:55.909581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:55.909870 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:55.910157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:55.910896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:55.911207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:55.912882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:55.926687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 269/538] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:37:55.932421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:56.062746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86d42795-154e-4d14-80c5-24e15e9a7c5d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:56.107809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:56.108674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:56.108967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:56.109256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:56.117561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:56.119043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:37:56.134917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:56.139042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:56.139863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 270/539] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:56.142418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:56.278219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34313824-e089-45c9-9712-e010fc729b4f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:56.321156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:56.321753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:56.322041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:56.322319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:56.323112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:56.323426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:56.324885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7670038-e03f-48f9-8300-fed85de1846e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:56.337513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 270/540] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:37:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:56.345001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:56.390629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:56.490156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae12777d-83f5-43af-a594-68efb0865bac None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:56.540059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:56.541111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:56.541608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:56.542082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:56.543122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=516b221cb1794430befadcde5f9fd77b) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:56.543631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:56.545277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b204914-9068-45ba-8c5e-ebc1472311f2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:56.555858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 271/541] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:37:56 2022] GET /identity/v3/role_assignments?user.id=516b221cb1794430befadcde5f9fd77b&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:56.735373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd023b00-cbdc-4677-8d02-3565dbbc2e04 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:56.735697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd023b00-cbdc-4677-8d02-3565dbbc2e04 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:56.735978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd023b00-cbdc-4677-8d02-3565dbbc2e04 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:56.737856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 271/542] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:56.744039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:56.744336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:56.744624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:56.769009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:56.770508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:56.869187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 272/543] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:56 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:56.902037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd023b00-cbdc-4677-8d02-3565dbbc2e04 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:56.939098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:57.040306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd023b00-cbdc-4677-8d02-3565dbbc2e04 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:57.084331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:57.085191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:57.085471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:57.085745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:57.098583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:57.107275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:57.108167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:57.108938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 272/544] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:56 2022] GET /identity/v3/roles/service => generated 85 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:57.115915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:57.168945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:57.273796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ff44ae-00e8-4415-859d-3e1fe7d95e0e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:57.330969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:57.331656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:57.331972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:57.332253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:57.333011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:57.333891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:57.335271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:57.350881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 273/545] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:57 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:57.356587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:57.484209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:57.494100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbb06854-1d4e-4a1b-819f-f0724622886f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:57.542022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:57.542746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:57.543025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:57.543295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:57.560126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:57.561594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:57.582630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:57.584130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:57.585040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:57.585040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:57.585040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:57.585040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:57.586598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 273/546] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:57.595973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:57.744677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:57.763550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53b5bd2e-5392-45bd-aaaa-17db2314d530 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:57.820570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:57.821231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:57.821525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:57.821810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:57.822545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:57.822858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:57.824262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2509234-f904-4799-a794-d56a653d9299 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:57.840472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 274/547] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 245 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:57.846002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:57.945061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6918f75-03a8-48c6-aa2c-6713ab73a780 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:57.945375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6918f75-03a8-48c6-aa2c-6713ab73a780 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:57.945620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6918f75-03a8-48c6-aa2c-6713ab73a780 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:57.947254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 275/548] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:37:57.965941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:57.966231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:57.966497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:57.979685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bfd5bc-44cc-4895-81e6-4568526f2c8c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.000932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:58.002145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:58.014715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.015519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.015782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.016091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.026936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:58.028552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:37:58.047744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:58.049929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:58.051484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 274/549] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:37:57 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:58.058596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.072303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.131940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 276/550] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:58.156803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.256297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6df12c-e02e-4c4c-86b0-45996db9a8b3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.282678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.294074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.295042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.295549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.295912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.296927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:58.297404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:58.298946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0964c5f0-90a1-4f13-90c9-0df97819ab4c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:58.312091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f08d30b4-0a21-41c0-a502-13d6de04fd69 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.318469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 275/551] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 260 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:58.324513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7dcbc-3090-4b30-9d7e-e86ccb0607d2 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.324810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7dcbc-3090-4b30-9d7e-e86ccb0607d2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.325164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edd7dcbc-3090-4b30-9d7e-e86ccb0607d2 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.326809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 276/552] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:58 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:58.327608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7dcbc-3090-4b30-9d7e-e86ccb0607d2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.359133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48573d42-5b75-4ba5-be7d-9f90797147cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.360411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48573d42-5b75-4ba5-be7d-9f90797147cd None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.361095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48573d42-5b75-4ba5-be7d-9f90797147cd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.361997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48573d42-5b75-4ba5-be7d-9f90797147cd None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.375579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48573d42-5b75-4ba5-be7d-9f90797147cd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-48573d42-5b75-4ba5-be7d-9f90797147cd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:58.387343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:58.388318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:58.389104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:58.391659 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 277/553] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:37:58.393319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.393852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.394255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.406248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.420074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:58.421644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:58.451408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edd7dcbc-3090-4b30-9d7e-e86ccb0607d2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.487492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.488181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.488470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.488749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.504282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 278/554] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:58.506295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.561985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:58.563717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:58.580012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.613624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd5187a-33b3-4f61-9e65-9ae0161a3d27 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.625403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.649504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.650432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.650860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.651280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.652225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:58.652699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:58.654071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebf809f1-72de-4201-8d21-271ba8d30b5a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:58.662578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 279/555] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:58.665134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe146f-22d0-4e99-9cbf-3ba3fa01ada1 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.665651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe146f-22d0-4e99-9cbf-3ba3fa01ada1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.666124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80fe146f-22d0-4e99-9cbf-3ba3fa01ada1 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.666564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 277/556] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:37:58.667468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.672028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 280/557] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:58 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:58.672945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe146f-22d0-4e99-9cbf-3ba3fa01ada1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.783186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b86c681-4f3f-423f-90e7-cd472cef98c3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.784740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80fe146f-22d0-4e99-9cbf-3ba3fa01ada1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:58.816597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.820978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.821974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.822288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:58.822586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.823005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.823198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.823382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.823632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.834337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:58.835705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:58.836358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:58.846486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:58.847442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:58.848274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:58.851999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 278/558] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/roles/service => generated 85 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:58.853160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-e722fc66-09ef-4089-85a9-ff1142d97b0f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:58.853632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:58.854989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:58.855830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:58.855830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:58.855830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:58.855830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:58.862524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 281/559] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:58.863599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:58.863907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:58.864192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:58.882307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:58.883962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:58.885618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:58.958742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 282/560] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:58 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:58.958742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:58.968516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05cb336f-0da6-4ff9-9aa7-2242701364bd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.007857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.008975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.009550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.009829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.021217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.022771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:59.041268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:59.042211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:59.043116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:59.043116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:59.043116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:59.043116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:59.046985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 279/561] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:59.047713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.073470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e9545ce-abc3-4ce4-82ff-b45b8d604fa4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.088013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.109894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.110563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.110846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.111121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.111883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:59.112201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.113633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.123382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 283/562] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:58 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:59.124204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.129862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.158881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45469d6-2c6a-4e15-a944-a2a3d5f8de98 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.227572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.228267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.228553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.228823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.229541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:59.229852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.231215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.242221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 280/563] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:59.243119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.258948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95de84bb-597a-4e44-bfce-c8eebf48a1fa None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.292320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.293121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.293403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.293680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.303417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:37:59.311137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:59.312230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 284/564] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:59.313093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.342395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd353a16-5272-471e-a76d-25806bd80b8d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.350668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.375627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.376317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.376602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.376878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.377601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:59.377910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.379296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b8cc1a-e09f-44fb-bcdb-bae088f04c11 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.380767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.391457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 281/565] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:59.394193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5ba32-b676-4379-ba2b-065d81d8fa7b None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.394193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5ba32-b676-4379-ba2b-065d81d8fa7b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.394193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b5ba32-b676-4379-ba2b-065d81d8fa7b None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.394766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 282/566] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:37:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:37:59.395383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5ba32-b676-4379-ba2b-065d81d8fa7b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.418373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4bed5b-9864-41a7-a99d-27e80815dd2d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.450568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.451353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.451642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.451949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.463287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.464747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:37:59.482084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:59.482953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:59.484039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:37:59.484039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:37:59.484039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:37:59.484039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:37:59.484039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 285/567] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:59.484039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.497146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b5ba32-b676-4379-ba2b-065d81d8fa7b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.531038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.531950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.532245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.532518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.538930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.540398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:37:59.553983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:59.554926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:59.555784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 283/568] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:59.556280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.568230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.582913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da6e521e-b78d-484c-9972-888c57d69b28 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.613437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.616251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.616926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.617213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.617488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.618230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:59.618540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.619932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60398d-e952-480d-b458-75cfb1141457 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.627576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 286/569] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:59.628784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.629080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.629400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.645408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:37:59.647168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:37:59.661095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f860f68-6031-4d0b-8c73-3adab7a3331b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.697212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.698010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.698288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.698557 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.705974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.707392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.724033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 287/570] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:37:59 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:37:59.724033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:37:59.724281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:37:59.725339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 284/571] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:37:59.726457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.800068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.821196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23dab794-cb89-4907-9278-5dd2e4e62436 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.825777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a536feed-73e5-4173-91ef-37c5c3688206 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.839910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:37:59.855660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.856358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.856641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.856919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.857641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:59.857972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.859289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:37:59.859429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.859982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:37:59.860271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:37:59.860550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:37:59.861337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:37:59.861650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:37:59.863061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:37:59.871322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 288/572] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:59.872079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.876150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 285/573] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:37:59.876967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:37:59.971003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd285d9-0438-49ba-bb41-d6813b85982e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:37:59.975413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f173ed53-ee6a-4148-96cd-170361a8173b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.003033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.003825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.004144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.004417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.010059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.010729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.011016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.011299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.012137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.012228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:00.012463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.013894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.016208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.017716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.030690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 286/574] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:00.031552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:00.034180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:00.035243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:00.036213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:00.036213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:00.036213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:00.036213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:00.036213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 289/575] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:37:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:00.040343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.062947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.131118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51ececc6-55f9-4d3a-abc3-31271186dbac None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.138152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ed12a2-39fd-430c-868d-8c2ccb8bd2f9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.164859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.165652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.165936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.166435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.170765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.171548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.171829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.172136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] PATH_INFO: `/v3/users/glance` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.176154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:00.179912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.181348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:00.185049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:00.185975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:00.186790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 287/576] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:00.187460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:00.196951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:00.197858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:00.198756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 290/577] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:00.199321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.251657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.277088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.285712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7937015-eefa-42f3-8729-730dfb94f763 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.299903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6dde385-891b-4340-8e3c-1e194e4d8471 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.319765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.320443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.320730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.321002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.321818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=7740886489ae4fe78a1d81cd12852750) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:00.322130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.323552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.343901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 288/578] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/role_assignments?user.id=7740886489ae4fe78a1d81cd12852750&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:00.343901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.344223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.344223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.344223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.344223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.346593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.348123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:00.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:00.368332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:00.369199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:00.369199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:00.369199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:00.369199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:00.369199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 291/579] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:00.369890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.455950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c056d648-6907-4c56-9516-31fb96236146 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.481347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d62e86ae-295f-4b36-b684-b656dee805d6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.483130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.499194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.500334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.500625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.501528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.501633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:00.501919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.503326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.518736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.519429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.519694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.520007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.520917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:00.521224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.522591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.526008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 289/580] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:00.531044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.535745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 292/581] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:00.535745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.550193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.636275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a21dc4-f348-43fd-b15d-495faa7aaf40 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.639751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f1651b-4828-4f5f-acdb-21b736b337a8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.671625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.672338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.672632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.672917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.673764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:00.674078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.675508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.683165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41978451-2c15-4340-8338-3318a3669877 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.684464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41978451-2c15-4340-8338-3318a3669877 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.684925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41978451-2c15-4340-8338-3318a3669877 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.686305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41978451-2c15-4340-8338-3318a3669877 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.687216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41978451-2c15-4340-8338-3318a3669877 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:00.687726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41978451-2c15-4340-8338-3318a3669877 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.689313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41978451-2c15-4340-8338-3318a3669877 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.697886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 290/582] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:00.697886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.704940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 293/583] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:00.707955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41978451-2c15-4340-8338-3318a3669877 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.739180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.774755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:00.799371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e51a745-95be-46b6-bf91-307f02038f9e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.818407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41978451-2c15-4340-8338-3318a3669877 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.832546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.833894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.844004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.844004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.844004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.844004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:00.858031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:00.858951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:00.859863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 291/584] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/users/cinder => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:00.861077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.864612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:00.865470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:00.865948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:00.866236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:00.880059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:00.881973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:00.902541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:00.903430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:00.904345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:00.904345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:00.904345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:00.904345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:00.904345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 294/585] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:00.904636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:00.974712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab58f56e-0f94-4c7a-8d77-5593ba25bb6f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:00.995696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.009920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.010722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.011005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.011282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.013454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.020054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eadd9a35-9a58-4347-987c-ddf2052aa361 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.023175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.024655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:01.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:01.042429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:01.043300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:01.043300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:01.043300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:01.043300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:01.043300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 292/586] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:01.043565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.060616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.061758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.062223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.062676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.071075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.072558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:01.087316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:01.088273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:01.089148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:01.092279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 295/587] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:01.093207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.151621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-570b544d-9782-47c3-9833-882fccf21f3e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.193815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.194497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.194784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.195038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3816e18-169f-4e68-830d-9b230a3a1995 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.195229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.195940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:01.196241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.198041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.220555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 293/588] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:01.220555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.235600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.236280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.236571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.236878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.237607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:01.237918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.239461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.242283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.263202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 296/589] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:01.263202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.269922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.327207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9332e251-6762-47c1-a6a2-d99202780f1f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.366619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.366803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb4a8848-7c89-4404-a1d8-5fa67d791fd1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.367362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.367571 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.367851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.368701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:01.369485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.370523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.387665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 294/590] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:01.387665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.409845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.410521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.410811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.411129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.411956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:01.412276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.413712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.425294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 297/591] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:01.426104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.488122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.494154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe9d360d-2a47-4e01-ab46-5660bf74dbc8 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.497326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.531364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.532196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.532512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.532757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.535987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be649d3c-7324-4f2b-a783-801a8b2452cf None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.542784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.544274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:01.562397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:01.563428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:01.569307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:01.569307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:01.569307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:01.569307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:01.569307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 295/592] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:01.569307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.579500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.580347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.580633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.580907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.588885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.590313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:01.604407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:01.605352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 298/593] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:01.611183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.672224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da5a6487-3052-4479-aca6-33e02d1e118d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.712961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.713755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.714038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.714480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.717119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bd7b6a9-8ed0-4893-9325-2095449aab5f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.721182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.722634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.733884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:01.736804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:01.737649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:01.738544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 296/594] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:01.739019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.754327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.755253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.755542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.755825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/54986964e91b4af7997e09d4682cc31e/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.762096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:01.771844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=54986964e91b4af7997e09d4682cc31e, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.773389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.842176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17be6afe-8aec-4fa2-ad8c-031d8de8f3f3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:01.859905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 299/595] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:01 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/54986964e91b4af7997e09d4682cc31e/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:01.859905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.880549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:01.881393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:01.881802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:01.882236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:01.883165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:01.883698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:01.885330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:01.898873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 297/596] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:01.899903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:01.992595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed5743fc-1753-4fba-bb3f-a9efadf5adc5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.009531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.035643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.037075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.037459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.037830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.040542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:02.040941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.042527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.047450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45a95ae-f7a4-4aa3-a498-f63ba96fae38 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.065554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 300/597] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:02.066414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.095062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.095734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.096052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.096331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.097120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:02.097430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.099379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.101005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.112970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 298/598] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:01 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:02.114052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.213339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fb895e1-6aa6-4a0c-ba1f-81cb8cda4194 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.233823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e91a526a-ebb0-470b-a948-dafc17022c60 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.257291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.258665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.259200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.259694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.267775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.269855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.274167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.274972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.275254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.275530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.278600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:02.287661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:02.294060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:02.294930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:02.295336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.295336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.296059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 301/599] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:02.297092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:02.309600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:02.310499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:02.311421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:02.311421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:02.311421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:02.311421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:02.311421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 299/600] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:02 2022] GET /identity/v3/domains/Default => generated 87 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:02.312178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.396929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.421963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50b83461-b8b2-4947-ba01-ab975ca216f3 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.447577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42aa139e-f43b-4c66-8106-f2e5a88e4bc6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.463269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fadfc35c-881f-4447-a828-483d0028c86f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.464576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc35c-881f-4447-a828-483d0028c86f None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.464963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc35c-881f-4447-a828-483d0028c86f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.465360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fadfc35c-881f-4447-a828-483d0028c86f None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/35d68b3bee5445849c59da135df06fea/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.481345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfc35c-881f-4447-a828-483d0028c86f None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=35d68b3bee5445849c59da135df06fea, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.482908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fadfc35c-881f-4447-a828-483d0028c86f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.487965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.488625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.488905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.489201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.489978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:02.490285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.491711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.503221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 300/601] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:02.504069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.556005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 302/602] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:02 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/35d68b3bee5445849c59da135df06fea/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:02.556005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbdc59-eb33-408d-97fb-4000580577fd None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.556005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbdc59-eb33-408d-97fb-4000580577fd None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.556005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dbdc59-eb33-408d-97fb-4000580577fd None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.556005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 303/603] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:02.556005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbdc59-eb33-408d-97fb-4000580577fd None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.562346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.622030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a595ad-ab09-4653-a9c4-9cefc9f83975 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.666255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.667096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.667457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.667914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.668774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:02.669122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.670708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-429ab74b-0c1c-4262-8e68-5c900911c2f5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.678360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dbdc59-eb33-408d-97fb-4000580577fd None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:02.694374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 301/604] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:02.695861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.696175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.696452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.718427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:02.718755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:02.719581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:02.719777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:02.719893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:02.720150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/72c24cb67566467dbdf65cfd44a59274/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:02.723116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.737492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=72c24cb67566467dbdf65cfd44a59274, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:02.739152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:02.799060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 302/605] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:02.799750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.840587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:02.851517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 304/606] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:02 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/72c24cb67566467dbdf65cfd44a59274/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 299 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:02.852462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:02.963147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c853ca3-c319-495f-bdad-7791210af109 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.029488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.030317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.030600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.030872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.044531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:03.046078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:03.055147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecf5fc67-8038-4c86-a047-2c15d1e87cdc None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:03.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:03.069991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:03.070839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 303/607] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 271 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:03.075838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:03.105456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.106175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.106463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.106739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.123096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:03.131773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:03.132698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:03.133511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 305/608] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:02 2022] GET /identity/v3/roles/service => generated 85 bytes in 281 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:03.139332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:03.225874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:03.247083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:03.254620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96d71ed-7178-48eb-932d-43ce2b73cc81 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.306667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-581af6dd-1dd4-4c20-ad1e-07af27330844 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.342494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.343451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.344023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.344597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.345517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:03.346005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:03.347547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:03.360918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.361605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.361900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.362222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.363198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:03.363553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:03.366298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:03.372522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 306/609] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 233 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:03.375778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:03.395548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 304/610] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:03 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:03.401541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:03.578249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf0a696a-e6e0-4448-9b31-d80a941c7f7a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.620517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66a36d4b-eeaa-4283-8c5a-b440a6fb691c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.628351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:03.662434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.663522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.664490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.664942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.669982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.671087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.672588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.678181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/516b221cb1794430befadcde5f9fd77b/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.684740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:03.686299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:03.705742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=516b221cb1794430befadcde5f9fd77b, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:03.707472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:03.725121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:03.728991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:03.729896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:03.729896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:03.729896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:03.729896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:03.729896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 307/611] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 351 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:03.735069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:03.806016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 305/612] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:03 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/516b221cb1794430befadcde5f9fd77b/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:03.852651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5c63364-f980-42e6-9b00-b9cb9553fa90 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:03.889108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:03.889707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:03.889996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:03.890280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:03.891074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:03.891390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:03.892797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:03.906417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 308/613] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:03.912807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:03.930414 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:04.052089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ac23092-d430-430a-beda-3a828ffd3cf4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:04.097611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:04.098408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.098723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.098997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.108674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:04.110172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:04.125191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:04.128304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:04.129242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 306/614] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:03 2022] GET /identity/v3/users/swift => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:04.133891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:04.159959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:04.254121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7726f99e-41dc-492e-935a-041b3b8bf92d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:04.309009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:04.310009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.310476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.310929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.311975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:04.312485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:04.314102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f6d69a-3344-431f-ba67-9257fd325ebe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:04.342087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 309/615] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 11:38:04 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:04.347737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:04.402697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38449d7e-6897-4a24-a3b5-80a4803b2b26 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.403304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38449d7e-6897-4a24-a3b5-80a4803b2b26 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.403770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38449d7e-6897-4a24-a3b5-80a4803b2b26 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.405329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 310/616] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:04 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:04.417246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.417466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.417745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.443960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:04.445872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:04.448302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:04.502125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efee236f-7aec-4d1b-bec0-0657ae422536 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:04.556245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:04.557635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.558531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.558997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.570965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:04.573107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:04.586184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 311/617] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:04 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:04.592257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:04.593468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:04.594328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:04.594328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:04.594328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:04.594328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:04.596345 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 307/618] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:04.605649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:04.615281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:04.616743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:04.786747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-524e6f76-7afa-49cc-89eb-8607a32df10c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:04.804065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a3d6bc-319b-418b-9ad0-bacab7959519 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:04.842945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:04.853979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:04.857085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.857085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.857085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.857085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:04.857085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:04.857648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:04.872918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:04.874387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:04.874673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:04.874952 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:04.882996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 312/619] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:04.887771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:04.888996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:04.895862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:04.896798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:04.897806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 308/620] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:04 2022] GET /identity/v3/roles/service => generated 85 bytes in 283 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:04.912127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.031298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.063641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22cc5fa7-1e07-46e3-9270-f642dd4bae5e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.075778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030823d6-3b5b-4ac3-adce-ecd350fd2406 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.101824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.102866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.103348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.103975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.105095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:05.105613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.107670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aee9a4f-df9d-4d67-b8d2-7cc0705ad95b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.121239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.121239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.121239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.121239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.122170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 309/621] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:04 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:05.123675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168b3e3-ce7c-4f92-a359-11d00d239177 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.124187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168b3e3-ce7c-4f92-a359-11d00d239177 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.124779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e168b3e3-ce7c-4f92-a359-11d00d239177 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.129735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.129735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.130184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 310/622] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:05.131153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168b3e3-ce7c-4f92-a359-11d00d239177 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-a0fc78d9-7444-4c9b-9cfb-d78690c3affe None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:05.143549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:05.144580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:05.146478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 313/623] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:05.147195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.147195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.147455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.172743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:05.174372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:05.202919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.255576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 314/624] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:05.256754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.286822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e168b3e3-ce7c-4f92-a359-11d00d239177 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.322904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.332424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.335799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.335799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.335799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.351803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.353945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:05.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:05.381587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:05.382474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:05.382474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:05.382474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:05.382474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:05.385453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e4feba-9fb9-474c-88f7-42fbb0e060fe None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.387232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 311/625] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 252 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:05.388515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.430824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.431499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.431783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.432093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.432881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:05.433196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.434621 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.446614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 315/626] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:05.447779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.493137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.499484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d622d3-69bb-4922-b116-acba0020e4ae None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.534045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e10d5347-6822-4c21-bc50-4e237fdb28dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.534840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d5347-6822-4c21-bc50-4e237fdb28dc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.535123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d5347-6822-4c21-bc50-4e237fdb28dc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.535402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e10d5347-6822-4c21-bc50-4e237fdb28dc None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.545336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e10d5347-6822-4c21-bc50-4e237fdb28dc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:05.553008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859903f7-2386-41f7-8fa0-be784a9ac901 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-e10d5347-6822-4c21-bc50-4e237fdb28dc None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:05.554186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:05.555115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:05.555970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 312/627] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:05.557283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d787b-b2b5-4552-8907-65834a6e7ea8 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.557283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d787b-b2b5-4552-8907-65834a6e7ea8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.557546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-867d787b-b2b5-4552-8907-65834a6e7ea8 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.559982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 313/628] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:05.559982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d787b-b2b5-4552-8907-65834a6e7ea8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.579182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.597823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.599254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.599532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.599811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.600554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:05.600937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.602295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.622411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 316/629] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:05.622411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.671277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-867d787b-b2b5-4552-8907-65834a6e7ea8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.713470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.714377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.714665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.714956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/7740886489ae4fe78a1d81cd12852750/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.723767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5830c3a-a411-4f1a-9ef9-f67de6682f8c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.730276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=7740886489ae4fe78a1d81cd12852750, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.731726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.758192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.758868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.759173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.759432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.760217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:05.760530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.761882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c773f26-b09c-4e2f-8749-ff5664fed361 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.770074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 317/630] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:05.771742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.772067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.772354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.788208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:05.789392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:05.811994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 314/631] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:05 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/7740886489ae4fe78a1d81cd12852750/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:05.811994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.817866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.831452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:05.877647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 318/632] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:05.878763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:05.940642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc7517af-efa0-4aa8-92c3-08617d8ab7cd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:05.980677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:05.981493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:05.981776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:05.982053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:05.992071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:05.993551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:05.997431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17e1f5f0-e188-4869-b6e0-3c8c238055d8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:06.010112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:06.011089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:06.012055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 315/633] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/users/nova => generated 82 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:06.012640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.034998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.035989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.036285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.036568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.047918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.049895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:06.069084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:06.070051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:06.070915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:06.070915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:06.070915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:06.070915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:06.075905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 319/634] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:06.075905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.093522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.099080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.133846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ddcd4c-85ff-4713-b48f-b533d88a4722 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.170318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.171410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.171851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.172153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.183390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:06.191329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:06.192949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:06.193779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:06.195429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbdc0263-1f8a-4f95-9f42-ff0265a03a31 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.198000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 316/635] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:06.198900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.244514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.245214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.245501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.245782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.246637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:06.247055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.249080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.273027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 320/636] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:06.274364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.316226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a129ef-eb4b-4542-b5f6-c8e8ecb2f6ec None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.341054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.355024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.356915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.357586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.358032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.358313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.359409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:06.360039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.361698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3f230b-0203-4c68-a8de-559675f4ebd8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.376859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79f6856d-9b4a-437d-8605-ffdd5ea57fe5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.379383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 317/637] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:06.380885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d11ed-ebca-4d14-a48a-3e5abe342730 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.381246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d11ed-ebca-4d14-a48a-3e5abe342730 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.381613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b8d11ed-ebca-4d14-a48a-3e5abe342730 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.383138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 318/638] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:06.387663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d11ed-ebca-4d14-a48a-3e5abe342730 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.417495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.418326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.418623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.419052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.419802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:06.420142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.421515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.436425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 321/639] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:06.436425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.493783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b8d11ed-ebca-4d14-a48a-3e5abe342730 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.527338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.528160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.528444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.528771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.542416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-642e78d4-bc98-444a-b27a-b6cf26cf7066 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.580538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.580764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.581341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.581731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.581923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] PATH_INFO: `/v3/users/glance` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.590221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.595482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.595482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.596698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.598138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-1d36b4c4-c3c4-4a5d-99fe-1a2fe8e4298b None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:06.607735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:06.608814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jun 16 11:38:06.609731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:06.612184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 322/640] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/users/glance => generated 84 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:06.613323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.613706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.614044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.631806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:06.633067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:06.706906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 323/641] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:06.707657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.730867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 319/642] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:06.732529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.810949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f6978-3062-4481-a2db-084cc8866cbf None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.817651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.833341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc194d-fe9c-4cfa-b1eb-7008c7222ea2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:06.842872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:06.849705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.850479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.850773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.851116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.860606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.862084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.868951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:06.869646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:06.873158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:06.873158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:06.873158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:06.873158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:06.873158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:06.879136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:06.880199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:06.881081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:06.881081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:06.881081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:06.881081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:06.881081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 324/643] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:06.881370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.897156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 320/644] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:06.898306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:06.993385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-919b2a16-05b4-4ad0-95ec-7358830eb203 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.004135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda908ef-12bd-44ed-b7a9-b2c1f0fa9d36 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.033513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.034315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.034602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.034884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.040215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.041008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.041291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.041562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.045349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:07.053350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:07.054264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:07.060054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:07.060880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.060880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.061038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 325/645] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:07.061038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.061757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:07.081107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:07.082396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:07.089052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:07.089052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:07.089052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:07.089052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:07.089052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 321/646] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:07.089052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.105035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.170080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6780a4-4065-4d47-87a3-b4131c7962a1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.199045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5917218a-d29c-4b73-ae01-37106e45b93c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.204293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.204965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.205252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.205532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.206267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:07.206578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.208589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.220491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 326/647] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:07.225118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.238256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.239010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.239395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.239699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.309169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.316886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.327722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.327846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e8e84f-e7e7-48aa-8617-815ecae174db None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.327846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.360019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.360670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.360960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.361245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.361985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:07.362294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.363650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.376876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 327/648] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:07.377736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.428891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 322/649] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:07.428891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.486468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eb37393-9663-4cf4-90ca-8e96de9d5dd8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.526452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.527364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.527656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.527967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.528683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:07.528990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.529250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.530366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.535213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f85cd53-9ae2-473f-9a91-b86e5d1ce2dd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.542391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 328/650] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:07.545719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.569825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.579605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.581958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.581958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.581958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.590175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.591637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:07.605816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:07.606759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:07.607678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 323/651] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/users/cinder => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:07.608899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.655398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3ac492d-f45d-49ff-99d0-747444b7d7f5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.700981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.701792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.702082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.702785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.714141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.715017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98d8351-9c73-4945-bcdf-ab457ae4f0aa None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.715586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:07.734537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:07.735419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:07.736331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:07.736331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:07.736331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:07.736331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:07.736331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 329/652] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:07.737500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.752617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.753417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.753715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.753977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.764671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.766126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:07.786331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:07.787323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:07.788272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:07.788272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:07.788272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:07.788272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:07.792710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 324/653] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:07.793544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.803973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.819444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:07.843197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0637e7-afe1-4cfb-951e-2c732f260774 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.880906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.881712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.882001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.882279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.888958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.890402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.898190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2949b65f-062a-42c1-9644-354478eb3d31 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:07.904112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:07.905166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 330/654] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:07.911768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:07.945871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:07.946765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:07.947222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:07.947508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:07.948375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:07.948689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:07.950273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:07.975227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 325/655] 10.222.0.40 () {56 vars in 1137 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:07.975227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.030309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-591ab348-57b6-44ce-8afe-217231177e6c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.039699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.064457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.065124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.065408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.067816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.067816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:08.067816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.068220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.070512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.076147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf614a02-ae10-481c-b677-6cc644e8c3db None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.080293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 331/656] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:08.080998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.110742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.111418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.111713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.112054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.112816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:08.113127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.115118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.126659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 326/657] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:08.127304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.185623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-484ef972-8535-415c-a537-1ec3a6efa693 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.220301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.220967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.221252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.221527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.222313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:08.222623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.224655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0ca4376-b96b-46e5-b591-2115522082e9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.228917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a044b025-4841-4c2f-b7ba-61a5d2388019 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.237232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 332/658] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:08.239193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11200b7-0c1c-4a3a-a7e7-9b83ae74b9a1 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.239489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11200b7-0c1c-4a3a-a7e7-9b83ae74b9a1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.239784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e11200b7-0c1c-4a3a-a7e7-9b83ae74b9a1 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.242520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 333/659] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:08.242520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11200b7-0c1c-4a3a-a7e7-9b83ae74b9a1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.253520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.264346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.265001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.265287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.265566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.298788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.333154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.334586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.342214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e11200b7-0c1c-4a3a-a7e7-9b83ae74b9a1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.374582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.375373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.375658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.375960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.383168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.384645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:08.399075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:08.400140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:08.401206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 334/660] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/users/neutron => generated 85 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:08.401892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.432387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 327/661] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:08.433214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.458611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.502042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66d38a6f-e61d-44e5-a59b-1c97cba655f8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.518081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.531308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33e781c-5f49-49d2-998b-674a63225e0e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.535487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.536365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.536654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.536934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.543488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.544960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-d78db1dd-1799-4932-991a-613b8f6f14de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:08.559201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:08.560133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:08.561241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 335/662] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/projects/service => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:08.561964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.562258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.562489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.568175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.568829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.569110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.569381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.570238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=54986964e91b4af7997e09d4682cc31e) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:08.570547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.571992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.577897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 328/663] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/role_assignments?user.id=54986964e91b4af7997e09d4682cc31e&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:08.578664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.580111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:08.581296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:08.659377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 336/664] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:08 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:08.660356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.699607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.716732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a29e7bef-89be-4553-897d-cfe2a0d9821a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.761685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.762380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.762647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.762919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.763759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:08.764096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.765509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.776444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b921428a-c9e2-4aec-ae40-22a521862c9a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.784266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.788253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 329/665] 10.222.0.40 () {56 vars in 1139 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:08.788253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.819442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.820339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.820988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.821276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.834069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.835518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:08.853838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:08.854785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:08.855674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:08.855674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:08.855674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:08.855674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:08.855674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 337/666] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:08.856312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.905493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd2c8a44-f5e4-48ba-85b6-dedd1a6c5e27 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:08.940704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:08.941368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:08.941650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:08.941923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:08.943273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:08.943584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:08.945065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:08.957018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 330/667] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:08.957861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:08.966879 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:08.967617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e400bc2-cd5b-4c69-aee3-3301f63f8530 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.003844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.005315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.005699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.005980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.016096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:09.023854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:09.024968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:09.026019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:09.029964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 338/668] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:09.030761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.048942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.063262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05963229-43ac-4a4d-9c9a-cd9edb29b84a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.099500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.100191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.100479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.100755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.131600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-886196e9-57a4-4b1f-92b3-eabc091be5d2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.168891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.169558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.169844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.170275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.171006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:09.171318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.172726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.177801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.179472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.189888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 339/669] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:09.189888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.190151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.270323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.290474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 331/670] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:08 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:09.291320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.297377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf9660b-d96c-49c7-98ad-6e9c2dd481d2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.332411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.333104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.333387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.333668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.334500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=35d68b3bee5445849c59da135df06fea) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:09.334817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.336280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.342596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 340/671] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/role_assignments?user.id=35d68b3bee5445849c59da135df06fea&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:09.344857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.407552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d88b97-d4c7-4e09-a10b-28e4bca59c2f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.417342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.456772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.457932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.458902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.459709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.460683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:09.461096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.463679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.476404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 332/672] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:09.476404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.490215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e6fff4-a176-4c21-9a71-3359c0aaa1b9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.535437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.536773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.537058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.537332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.544823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.546382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:09.564022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:09.565008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 341/673] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:09.566177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.571214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.606907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-975a39b6-3217-42bb-a924-4abca4d4b9ec None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.646303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.647106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.647391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.647669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.660144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.662144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:09.686157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:09.687256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:09.688268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:09.688268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:09.688268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:09.688268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:09.693429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 333/674] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:09.694290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.708103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ceabc7a-bdfc-48bf-ad6d-d357f3957938 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.722698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.751952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.752761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.753078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.753358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.764815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.766573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:09.784560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:09.785764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:09.787393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:09.787393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:09.787393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:09.787393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:09.792519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 342/675] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/domains/Default => generated 87 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:09.793543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.829573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-705926dc-697b-4fb0-8191-875b69056888 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.853223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:09.877060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.878277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.878885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.879439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.880415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:09.880913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.883021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.899091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 334/676] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:09.900718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.903915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a489f5a-0817-40dc-baf2-a227d59e0b21 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:09.950677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:09.951594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:09.952202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:09.952667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:09.954032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:09.954593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:09.956321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:09.972966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 343/677] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:09.974637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:09.994924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:10.014983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02961b3a-7514-4538-a5d4-614f2a2ba90d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.051335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.052103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.052481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.053038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.053673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:10.054076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.055622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.072039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 335/678] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:10.073002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:10.103391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-683707e7-1bc3-46a2-af35-f084d2bef5f4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.121652 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:10.149208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.150231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.150663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.151102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.152105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=72c24cb67566467dbdf65cfd44a59274) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:10.152615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.154274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.168286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 344/679] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:09 2022] GET /identity/v3/role_assignments?user.id=72c24cb67566467dbdf65cfd44a59274&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:10.171149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:10.204265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bd31be8-d04d-4c17-865a-c6ac21b3bbd2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.300244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.301169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.301574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.301840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.321171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.322666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.327326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e5ca4f-c5e5-4342-9984-415135af7501 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.332244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:10.356958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:10.357908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:10.358851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 336/680] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:10 2022] GET /identity/v3/projects/service => generated 88 bytes in 286 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:10.365745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:10.380669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.381809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.382241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.382918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] PATH_INFO: `/v3/users/swift` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.396026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.397726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:10.418632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:10.419602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:10.420611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 345/681] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:10 2022] GET /identity/v3/users/swift => generated 83 bytes in 249 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:10.426001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:10.468665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:10.496120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-843f230f-1b8f-4c76-b8b2-1e966b122d9c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.532775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.533138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d64a412a-eba7-4fa4-95c4-f12ac0609abe None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.534023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.534512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.534990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.536082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:10.536618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.538306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.550975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 337/682] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:10 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:10.560009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:10.576978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.578347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.578836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.579533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.581425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:10.581948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.584377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.613372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 346/683] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 11:38:10 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:10.623892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:10.637338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:10.723911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69e75311-7d4d-43b3-88d3-98ee38b52f83 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.775235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.775932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.776261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.776535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.777434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=516b221cb1794430befadcde5f9fd77b) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:10.777749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:10.779229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0ba1b2-971a-4a3b-8d1a-128ca1c9a5e0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:10.791994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 338/684] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:10 2022] GET /identity/v3/role_assignments?user.id=516b221cb1794430befadcde5f9fd77b&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:10.809850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5687f-7ea8-482f-a193-9736fc62a2d4 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.810261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5687f-7ea8-482f-a193-9736fc62a2d4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.810491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e5687f-7ea8-482f-a193-9736fc62a2d4 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.813058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 339/685] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:10.817896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.818187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.818458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:10.831501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:10.838752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6e0ac7-6ac9-47e7-bf17-089fe6b37d0c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:10.853912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:10.855111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:10.880162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:10.881582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:10.882093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:10.882589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.000783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 340/686] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:38:11.018635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.020264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.026831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.097998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.127918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.135141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 347/687] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 516 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:11.141067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.148113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1c6a30b-8a43-4958-8221-b2e24e0c1e44 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.185390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.186200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.186534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.186812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.198413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:11.206732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:11.208227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:11.209060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:11.212514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 341/688] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:11.214880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.254948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8790b174-4dfa-49c7-868b-0134271def83 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.292582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.293745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.294257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.294740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.306883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.308779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.322941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25bce921-0426-4ac5-b38f-fa3d484e26c4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:11.327782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:11.328865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:11.329966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:11.329966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:11.329966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:11.329966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:11.332122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 348/689] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:11.338008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.339747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.362039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.362636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.362915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.363234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.364365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:11.364492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.365945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.379794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 342/690] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:11.381089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.393819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.473881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8bee8e-82f2-42df-90c4-14e5892fac42 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.487844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cce8cb-5839-44bf-988b-d531b49851af None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.523895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.524653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.524940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.525217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.536996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.538672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:11.556059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:11.556935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:11.557815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:11.557815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:11.557815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:11.557815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:11.559857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.560887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.561384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.561755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 343/691] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:11.562088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.562947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.563238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:11.563792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.565336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.596254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.598737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 349/692] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 261 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:11.608839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.678819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd79ba5-10af-44bc-8a97-68461a9a46cc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.720901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.721738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.727914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.727914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.727914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:11.727914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.727914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.742865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 344/693] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:11.744359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.840475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75045bfd-cf2e-44ab-a921-83fc0752d55f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.848559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.850723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:11.860588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db7acb8f-ee91-4389-90dc-8c6474afbd80 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:11.879002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.880758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.881661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.882123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.889515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.891197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.900487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:11.901593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.901974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.902337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-371b66db-2dfc-437d-a261-e4d6f8b22e34 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:11.906964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:11.907807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:11.908732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:11.911174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:11.911413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 350/694] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 303 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:11.912910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:11.913022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f474-4e23-48a0-9374-2826d80d228c None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.913498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f474-4e23-48a0-9374-2826d80d228c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.913989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c0f474-4e23-48a0-9374-2826d80d228c None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.915668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 351/695] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:11.916711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f474-4e23-48a0-9374-2826d80d228c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-7cb43516-035b-434e-bc3c-84fda2d9c701 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:11.932139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:11.933079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:11.934020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:11.938010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 345/696] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:11.939791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:11.940117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:11.940378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:11.963154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:11.964743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:12.036512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 346/697] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:12.037693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.051050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c0f474-4e23-48a0-9374-2826d80d228c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.093787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.094779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.095294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.095794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.096814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:12.097330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.099004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.112223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 352/698] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:12.113292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.138362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:12.145109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:12.166913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd39cb1-dd20-4f37-b763-be9b816017a0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.215238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.216044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.217181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.217627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.218484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:12.218800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.220278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.233723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9948f027-5459-432c-a6fb-cc006ac6ac9f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.244251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 347/699] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:12.245398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.275919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.276733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.277021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.277298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.288693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.290202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:12.309407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:12.310319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:12.311638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:12.311638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:12.311638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:12.311638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:12.311638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 353/700] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:12.311638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.357598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22785d2a-ca14-40bb-bf94-b7cd6a163807 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.393245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.394495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.395040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.395523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.396330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:12.397171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=7740886489ae4fe78a1d81cd12852750) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:12.397708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.399332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.402684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:12.407055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 348/701] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/role_assignments?user.id=7740886489ae4fe78a1d81cd12852750&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:12.413967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.416852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60452425-8c87-40c2-84a5-1d1ff6a7da72 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.459144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.459829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.460157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.460444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.520647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e925fc7f-a506-460b-be03-3cf762f186e9 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.537787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.539261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.558960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.560028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.560513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.560989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.561936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:12.562479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.564104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.581774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 349/702] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:12.583407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.630402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:12.648421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:12.652335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 354/703] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:12.659988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.696184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3d77803-5dc5-4a43-ac8f-ae10fa7d5665 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.733171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.734210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.734669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.735108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.736003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.737583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.768250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe50685-eba8-429c-8b5b-d503d718bef1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.811399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:12.812640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:12.812763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:12.813132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:12.826864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:12.828371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:12.846856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:12.847811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:12.849375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:12.849375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:12.849375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:12.849375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:12.849623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 350/704] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:38:12 2022] POST /identity/v3/users => generated 254 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:12.849705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 355/705] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:12.854715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:12.977435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bedf8d3-3a09-44e0-bea3-6c0d03d362f4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:12.998560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:13.019645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.020923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.021205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.021479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.022229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:13.022541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.024100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d006f8b-449e-482c-8445-d7bd04af6759 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.044562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 351/706] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:13.049813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.092159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:13.097622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cd9f4-11f8-43bf-b9e9-3b2b08784e4c None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.102841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cd9f4-11f8-43bf-b9e9-3b2b08784e4c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.102841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd2cd9f4-11f8-43bf-b9e9-3b2b08784e4c None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.102841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 352/707] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:13.105636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.105924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.106200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.129753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:13.130976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:13.175325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f580991-9389-4fd6-9175-98376b22b3a4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:13.233688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.234872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.235337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.240073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 353/708] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jun 16 11:38:13.240201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.243768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.245282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.259400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:13.266967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:13.267830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:13.268726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:13.272229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 356/709] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 223 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:13.279717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.410444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b56ea3-080d-46c7-bf47-3d001eb0570b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:13.420617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:13.481323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.482174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.482463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.482740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.483408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.484828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.491424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:13.502045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3048cf2c-4b55-462d-90ef-30e556db6556 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:13.513765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:13.514735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:13.516014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 354/710] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 256 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:13.522944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.540551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.541554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.542055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.542571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.543581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:13.544102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.545694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed70ecf-9e88-41fb-9e87-cb7ff33a1865 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.558864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 357/711] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:13.560648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5f3ef-b6a0-4ffb-bf8f-c4f174fa602d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.561102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5f3ef-b6a0-4ffb-bf8f-c4f174fa602d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.561541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18f5f3ef-b6a0-4ffb-bf8f-c4f174fa602d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.567544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 358/712] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:13.568572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5f3ef-b6a0-4ffb-bf8f-c4f174fa602d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.656620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4853268e-1866-460f-a54c-a5a1a8e257cb None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:13.693079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18f5f3ef-b6a0-4ffb-bf8f-c4f174fa602d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:13.703331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.703946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.704225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.704501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.705205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:13.705510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.706931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f675adb-9269-478a-a627-5bc99a22fcb8 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.716007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 355/713] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:13.717348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.717637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.717919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.736119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:13.737307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:13.738185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.740016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.741403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.741926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.743013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=54986964e91b4af7997e09d4682cc31e) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:13.743526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.745236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.754136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 359/714] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/role_assignments?user.id=54986964e91b4af7997e09d4682cc31e&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:13.755159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.767645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:13.775978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:13.820062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 356/715] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:13 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:13.838473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:13.866542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98fb2e42-a675-42e1-844e-ba3b8bf0c20c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:13.902520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:13.903656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:13.904201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:13.904680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:13.905591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:13.906102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:13.907810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:13.915082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 360/716] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:13.980368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1422a031-b70c-4292-af72-51a0b0d63419 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:14.037801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:14.039005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:14.039541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:14.040090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] PATH_INFO: `/v3/services/network` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:14.040965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:14.042589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:14.059639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:14.060868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:14.061682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:14.064808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 357/717] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:13 2022] GET /identity/v3/services/network => generated 88 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:14.070527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:14.196028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:14.224101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62598798-5de8-4c7a-8371-bd7d637509e6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:14.268327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:14.268929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:14.269211 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:14.269488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:14.270794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] RBAC: Adding query filter params (name=network) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:14.271102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:14.272564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9756239-f45d-4318-96aa-397c65391c9a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:14.279493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 361/718] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:14 2022] GET /identity/v3/services?name=network => generated 124 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:14.286129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:14.325287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:14.416604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb21341-42bd-4d55-81ee-b57f8d18d016 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:14.464018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:14.464712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:14.465000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:14.465278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:14.466071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] RBAC: Adding query filter params (type=network) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:14.466387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:14.467774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eab95692-e095-4d5a-8c03-b5d7861db08a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:14.478979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 358/719] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:14 2022] GET /identity/v3/services?type=network => generated 124 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:15.154391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ac66-1ca0-4867-abb0-84e18c671472 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:15.154951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ac66-1ca0-4867-abb0-84e18c671472 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:15.155030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b2ac66-1ca0-4867-abb0-84e18c671472 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:15.156910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 362/720] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:15.163112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:15.164017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:15.164498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:15.188422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:15.189934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:15.280037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 359/721] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:15.304941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2ac66-1ca0-4867-abb0-84e18c671472 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:15.447240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b2ac66-1ca0-4867-abb0-84e18c671472 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:15.496318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5de114-a4c1-4819-9efa-64b56a462923 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:15.497175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5de114-a4c1-4819-9efa-64b56a462923 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:15.497465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5de114-a4c1-4819-9efa-64b56a462923 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:15.497743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5de114-a4c1-4819-9efa-64b56a462923 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:15.513541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b5de114-a4c1-4819-9efa-64b56a462923 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5b5de114-a4c1-4819-9efa-64b56a462923 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:15.522074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:15.523036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:15.523861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:15.527970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 363/722] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:15.531606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:15.583952 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:15.631374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccfeae-589d-4416-a5ad-b9e62173a02c None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:15.631681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccfeae-589d-4416-a5ad-b9e62173a02c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:15.633200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35ccfeae-589d-4416-a5ad-b9e62173a02c None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:15.635017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 364/723] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:15.639670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:15.639991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:15.640275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:15.648071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:15.664915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:15.666136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:15.724424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bc374c-4cc2-48b7-9dd9-cfa6cc53ef2a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:15.791001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:15.791692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:15.792008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:15.792292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:15.793049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:15.793363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:15.794746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:15.799133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 365/724] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:15.811710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 360/725] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 282 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:15.817549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:15.851161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:15.986719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2aa9c19-1d4e-44c6-bd48-7c35093a31d2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.004331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.011006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb9f1f9-1338-4726-9dfe-be2a95fa29aa None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.052113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.052935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.053386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.053802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.065311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:16.074477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:16.075488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:16.076606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 361/726] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:16.084084 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.090467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.091287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.091574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.091854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.108230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.117799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.119272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:16.149844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:16.151040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:16.162141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:16.162141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:16.162141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:16.162141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:16.162141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 366/727] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:16.168944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.208578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f26249c-f4f8-4b9b-a9c4-c9744f56ae57 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.246889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.247915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.248412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.248961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.249966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:16.250418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.252027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.265616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 362/728] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:16.267010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.276930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.330927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da3800c-964f-4796-9051-24becb73ba28 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.376369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d05cecc-b7f0-4bc4-a0f2-a41f0f5ba095 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.413401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.414442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.414901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.415321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.417932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.418601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.418881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.419509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.419922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:16.420238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.421669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.429600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.431366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:16.450845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:16.452011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:16.453097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:16.453097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:16.453097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:16.453097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:16.453314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 367/729] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 283 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:16.459589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 363/730] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:16.463097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.466799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.533874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.544937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.589353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83f212fe-664d-427e-8327-6bb2e51bec4b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.616435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6a943d-ec73-4f81-8776-88d2a9c3b1c3 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.627175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.627593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.628001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.628338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.629211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:16.629611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.631234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a16d324-67c9-4ac4-bfaf-3552e5bfcfb7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.650735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 364/731] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:16.652493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00833b9-a359-41f8-a5c7-d8d38a4f8575 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.652856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00833b9-a359-41f8-a5c7-d8d38a4f8575 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.653382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00833b9-a359-41f8-a5c7-d8d38a4f8575 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.654870 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 365/732] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:16.655694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00833b9-a359-41f8-a5c7-d8d38a4f8575 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.663497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.664714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.665981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.666429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.676835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.678697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-6d2cc293-0171-4c4b-8d9a-67109de9f296 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:16.698833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:16.699985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:16.701022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:16.703935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 368/733] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/users/swift => generated 83 bytes in 238 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:16.705340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7c3a-2908-40ee-971d-8a3c754ee3fc None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.706278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7c3a-2908-40ee-971d-8a3c754ee3fc None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.706563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623c7c3a-2908-40ee-971d-8a3c754ee3fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.725106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-623c7c3a-2908-40ee-971d-8a3c754ee3fc None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:16.726291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-623c7c3a-2908-40ee-971d-8a3c754ee3fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:16.780575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00833b9-a359-41f8-a5c7-d8d38a4f8575 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.797683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 369/734] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:16.798940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f821c-4866-4280-b523-266e752ae6bc None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.799245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f821c-4866-4280-b523-266e752ae6bc None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.799505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a0f821c-4866-4280-b523-266e752ae6bc None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.800997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 370/735] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:16.801517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f821c-4866-4280-b523-266e752ae6bc None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.805547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.825303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.826785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.828636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.828636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.836815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.838653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.850776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-84bb344c-2157-431d-a271-4c167d41245a None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:16.854334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:16.855330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:16.856322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:16.859121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 366/736] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:16.864372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.864372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.864372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.880567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:16.881748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:16.929791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a0f821c-4866-4280-b523-266e752ae6bc None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:16.956037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 367/737] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:16.956037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:16.968250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:16.968928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:16.969214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:16.969487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:16.970457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:16.970770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:16.972226 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:16.990251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 371/738] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:16.991083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.076384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c8e2cc0-0e05-43bb-8264-bb6aa6274baa None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.086854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.088564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.095066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6d102c-f223-41d8-827c-ec30ce5e03a0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.115262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.116626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.117275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.117636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.131854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:17.133035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.133719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.134003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.134281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.135130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:17.135439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.136908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:17.140184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:17.141121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:17.141974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:17.146447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 368/739] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:17.147307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.158771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 372/740] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 11:38:16 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:17.159560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.252423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1229ea50-7e57-40ef-bcc5-4ca7a8ec4cf3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.258376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a90275a5-fa99-476d-8656-b2d037c0de31 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.285843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.286526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.286819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.287105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.287832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.289253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.291688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.292377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.292662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.292943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.314358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.358690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.360086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.361807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 369/741] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:17 2022] POST /identity/v3/services => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:17.362533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.397373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.483115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 373/742] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 319 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:17.483115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.502794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e2eef0-5b80-408d-862f-c12d39c30fec None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.540393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.544544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.545223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.545511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.545791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.546544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:17.546857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.549089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb12bd70-2fba-4702-bbe3-9b48c9fc2b58 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.559487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 370/743] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:17.560883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f980310-0edb-4eb8-887d-482909e2a85e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.561181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f980310-0edb-4eb8-887d-482909e2a85e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.561466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f980310-0edb-4eb8-887d-482909e2a85e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.562788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 371/744] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:17.563486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f980310-0edb-4eb8-887d-482909e2a85e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.596571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d09a61d3-23b2-43e6-b76c-d1539fff4a53 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.632301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be499aa8-66d5-4f64-a18d-e0396f2db7e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.633003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499aa8-66d5-4f64-a18d-e0396f2db7e6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.633312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499aa8-66d5-4f64-a18d-e0396f2db7e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.633616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be499aa8-66d5-4f64-a18d-e0396f2db7e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.657312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.677289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f980310-0edb-4eb8-887d-482909e2a85e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.707077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be499aa8-66d5-4f64-a18d-e0396f2db7e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.708632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be499aa8-66d5-4f64-a18d-e0396f2db7e6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.715782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.716725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.717012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.717292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.731327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.732809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:17.750623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:17.751502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:17.752411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:17.752411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:17.752411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:17.752411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:17.752411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 372/745] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:17.753254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.768723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.826315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 374/746] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jun 16 11:38:17.826315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.826315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.826315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.842210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:17.843530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:17.861486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14778eaa-97a8-4cd8-b2c8-22372e76fc6e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:17.898994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:17.900099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:17.900626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:17.901042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:17.908908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:17.923889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:17.924061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 375/747] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:17 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jun 16 11:38:17.924061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:17.924252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:17.933747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:17.934630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:17.936102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:17.936102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:17.936102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:17.936102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:17.936102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 373/748] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:17.936394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.011502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.030347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a384d9a9-2f6d-457d-aff6-bf409e228cec None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.042201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-084d3fd8-b486-495d-a8bf-e63ca93cb1f0 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.068808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.069473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.069904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.070189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.070903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:18.071209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.072594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.076173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.076966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.077249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.077525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.088618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 376/749] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:18.088618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.093713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.095162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:18.113480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:18.114389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:18.115294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:18.115294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:18.115294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:18.115294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:18.115294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 374/750] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:18.115575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.162613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.198191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b807ec37-355e-4e24-99ca-2d17b5a7914f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.228801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fc134e5-0815-446b-958e-15e063039500 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.233308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.233973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.234253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.234524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.235252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.236731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.261771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.262433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.262718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.262999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.263711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:18.264050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.265393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.277372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 375/751] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:18.278173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.311483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 377/752] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:18 2022] POST /identity/v3/services => generated 240 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:18.312575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.320672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.400478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2d5ed4-0e4f-42e9-9c21-704ef56140c4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.402690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.446198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.446885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.447256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.447602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.455806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.458034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.458597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c674d6bf-c37e-4990-abd2-9b1ee511e00d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:18.476368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:18.477536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:18.478621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:18.479782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 376/753] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 202 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:18.485083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.503601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.505100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.505551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.506441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.507731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:18.508220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.510248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.523927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 378/754] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:18.524962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.595744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.614515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20368fbc-b30a-4735-bd46-7eabf6ff43e7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.649656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dfe5b6c-e286-485d-a7db-b16f45ee9cd5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.652946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.653160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.653513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.653859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.661958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.663597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:18.681628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:18.682778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:18.683931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:18.690012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 377/755] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/users/nova => generated 82 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:18.690785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.707606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.708554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.709236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.709758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.711300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:18.711857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.714420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.715513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.731937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 379/756] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:18.743258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.811054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca82901c-5cc0-4980-87fd-47f5afb76a48 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.851676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301115d8-3699-4d7b-9d78-0d3a6d17bece None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:18.858197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.859699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.860159 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.860578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.870841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.872478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.876462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:18.889257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:18.891234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:18.891234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:18.891234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:18.891565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:18.892582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:18.893501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:18.893902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:18.893902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:18.895199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:18.896258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 378/757] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/projects/service => generated 88 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:18.897423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.916611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 380/758] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:18.920723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:18.998608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:19.027205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ac4b1e9-a7c4-4432-87c9-dabaa732cb94 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:19.028452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-604f3b33-94ef-4bc3-acc9-219e85a2330d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:19.064903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:19.065627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.065978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.066325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.067264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=7740886489ae4fe78a1d81cd12852750) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:19.067645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:19.069208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:19.074554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:19.075466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.075952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.076058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 379/759] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/role_assignments?user.id=7740886489ae4fe78a1d81cd12852750&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:19.076524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.077474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:19.077954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:19.079572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:19.080861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:19.095234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 381/760] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:19.103919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:19.210611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:19.239617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3533a58-307b-4007-a08a-e1cc4e81db7a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:19.286852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:19.288740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.289201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.289647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.310766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:19.313940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:19.320160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:19.340447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72dd8a88-475b-4b2a-bd87-fbc77d87b2ec None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:19.352299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:19.353221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:19.354116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:19.354116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:19.354116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:19.354116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:19.354116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 380/761] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 277 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:19.359403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:19.410141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:19.411424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.411709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.412012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.412848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=2e7985c45e3e478cac56a7d939e3fe2a) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:19.413161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:19.414593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:19.420976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 382/762] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:19 2022] GET /identity/v3/role_assignments?user.id=2e7985c45e3e478cac56a7d939e3fe2a&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 320 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:19.530700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa444a93-c2c1-48fe-a7bf-60c02d09a76c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:19.585240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:19.585841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.586127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.586406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.587400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:19.587709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:19.589162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9998e3cd-fc88-4d27-b299-0d2d386afad3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:19.610894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 381/763] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:19.616866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:19.695260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698be0a9-5506-4b15-81b5-18701d4a2da5 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.695551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698be0a9-5506-4b15-81b5-18701d4a2da5 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.695833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-698be0a9-5506-4b15-81b5-18701d4a2da5 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.697216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 382/764] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:19.702839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.703363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.703827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.740905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:19.742155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:19.760426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2c3f0fe-7949-4c86-b062-a5a1dadfa0a2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:19.764628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:19.827266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:19.828098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:19.828379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:19.828651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:19.844295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:19.845974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:19.847499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:19.873609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:19.874548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:19.875401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:19.876723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 383/766] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:19 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:19.877921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 383/766] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:19 2022] GET /identity/v3/projects/service => generated 88 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:19.881414 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:19.909093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:20.057849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9184e4df-e8f8-4302-ad4f-5ce240e2e7bc None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.081938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81838bae-3d79-48aa-80e1-954970b900b4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.128401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.128401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.128401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.128401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.128871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:20.128871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.128871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.150730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 384/767] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:19 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:20.152231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.153279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.153782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.154264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.155209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.156960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.160655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:20.181497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:20.182449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:20.183264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:20.194320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 384/768] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jun 16 11:38:19 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 285 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:20.198631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:20.252231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:20.268234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:20.348887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435ebbc5-137b-4796-93bb-ca445eeae13d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.391219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff72702c-9958-4335-acf6-eff1827154e8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.402635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.403546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.403829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.404130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/54986964e91b4af7997e09d4682cc31e/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.409546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=54986964e91b4af7997e09d4682cc31e, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.411023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.440603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.441325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.442293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.442606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.443374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:20.443687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.445143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.450364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 385/769] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 11:38:20 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:20.455323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:20.514289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 385/770] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:20 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/54986964e91b4af7997e09d4682cc31e/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:38:20.556769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:20.567340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0ccc396-2dfb-44ef-8bcc-5132c45fd788 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.605393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.606229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.606558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.606882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.607606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:20.607930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.609352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c78b29-aa0d-41a8-8222-68bcebcc0036 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.615227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 386/771] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 11:38:20 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:20.621252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:20.778749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ae1329-346c-467f-893b-7ff882d6a01d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.782124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:20.793790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6e2be-31fb-4b03-adb4-bf27bdfeaa1b None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.794085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6e2be-31fb-4b03-adb4-bf27bdfeaa1b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.794344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f6e2be-31fb-4b03-adb4-bf27bdfeaa1b None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.798255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 387/772] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:20.805269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.805269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.805269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.820591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.821269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.821556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.821834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.822632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] RBAC: Adding query filter params (interface=public, service_id=a7db3051eac94901863b867fcbae463f, region_id=RegionOne) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:20.822949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.824512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.831065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:20.832541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:20.835023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 386/773] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:20 2022] GET /identity/v3/endpoints?service_id=a7db3051eac94901863b867fcbae463f&interface=public®ion_id=RegionOne => generated 194 bytes in 214 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:20.840708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:20.943836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5d136d-d394-4190-8828-fefd2dc6f777 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:20.979177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:20.983892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:20.983892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:20.983892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:20.983892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:20.983892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:20.987995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 387/774] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:20 2022] GET /identity/v3/services => generated 772 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:21.000137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 388/775] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:20 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 197 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:21.023252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.058930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:21.133074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16298f9a-b47d-4015-a28f-55dcda744de0 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:21.171239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:21.172061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.172352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.172633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] PATH_INFO: `/v3/services/network` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.173303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:21.174698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:21.187648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:21.188756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:21.190088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 388/776] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/services/network => generated 88 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:21.197597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.312603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:21.328948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5bf092-58da-48e8-8e01-6d5629796f54 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:21.367178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:21.367898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.368193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.368471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.369213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] RBAC: Adding query filter params (name=network) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:21.369525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:21.370982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5096002-95cf-4242-bab0-79d8d892d561 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:21.376710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 389/777] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:21.382868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.433122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3705f-f811-4e17-a03a-9587904efb7b None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.433459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3705f-f811-4e17-a03a-9587904efb7b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.433679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8f3705f-f811-4e17-a03a-9587904efb7b None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.435892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 390/778] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:21.441171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.442160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.442502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.464282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:21.465855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:21.531424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:21.546748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d2505e9-7967-43dd-b545-40f56f627216 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:21.552695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 391/779] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:21 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:21.586550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.590410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:21.595653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.597973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.599453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.600660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] RBAC: Adding query filter params (type=network) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:21.601915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:21.603470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:21.614574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 389/780] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/services?type=network => generated 350 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:21.621236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.728422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:21.734534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-445d02c4-fd27-4889-8bca-70c8417d9e32 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:21.758983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2b4e3ef-47fe-4b2a-919e-0add5e9fb6d6 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:21.772211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:21.773153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.773667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.774171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.775583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] RBAC: Adding query filter params (interface=public, service_id=0fec88abb8e142e2a37c765d0e22268e, region_id=RegionOne) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:21.776166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:21.777739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:21.785157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 390/781] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/endpoints?service_id=0fec88abb8e142e2a37c765d0e22268e&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:21.794125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.824640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:21.865150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:21.865976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.866244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.866518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.900503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:21.908280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:21.909219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:21.910060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:21.912491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0013a21c-4e22-4f47-a825-5b6b01751491 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:21.920041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 392/782] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/roles/admin => generated 83 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:21.924933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:21.956651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eac048d2-f08c-46c2-b467-d3af81d62a17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:21.957809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac048d2-f08c-46c2-b467-d3af81d62a17 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:21.958319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac048d2-f08c-46c2-b467-d3af81d62a17 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:21.958823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eac048d2-f08c-46c2-b467-d3af81d62a17 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:21.959789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac048d2-f08c-46c2-b467-d3af81d62a17 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:21.962273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eac048d2-f08c-46c2-b467-d3af81d62a17 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:21.969178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 391/783] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/services => generated 772 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:22.005593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ce7cb-6706-463f-be13-f80ab5b73862 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.006234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ce7cb-6706-463f-be13-f80ab5b73862 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.006719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00ce7cb-6706-463f-be13-f80ab5b73862 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.008483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 392/784] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:22.014132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.014770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.015298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.041911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:22.043631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:22.131505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 393/785] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:22 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:22.157409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.164023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f794ba5-a941-4a90-bf8c-76722a5a5ce4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.181051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:22.206551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.207614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.208166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.208645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.209626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:22.210136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:22.211787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:22.222370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 393/786] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:21 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 298 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:22.228119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.231113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:22.275912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73835c5c-076c-4e8e-a61f-4107232bca95 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.318837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.319652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.319981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.320262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.340544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-804dedc0-b99c-447f-bf9b-0bd7c901cbbd None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.341313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:22.361524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:22.362481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:22.363847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 394/787] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/roles/service => generated 85 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:22.369373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.381152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.382245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.382730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.383231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.395290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:22.397049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:22.415433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:22.416417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:22.417509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:22.417509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:22.417509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:22.417509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:22.419368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 394/788] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:22.431829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.478464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:22.502826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7d14a42-8407-489d-80b9-92a3efcfaa3a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.528381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:22.540265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.541860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.542861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.543442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.546827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:22.548284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:22.550788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:22.564556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 395/789] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:22.566111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.610897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b36b91d2-3804-4688-b73e-7d31f436c6fc None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.683404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.687555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.688031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.688454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.690405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:22.690854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:22.692829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:22.712410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 395/790] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:22.715103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.777921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43238df7-c15f-4b0e-a37b-d8f49dca5478 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.858595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e95b08-8d3a-4ed3-97e5-5ca4cfb3e9d1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:22.860354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.861401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.861888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.862392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.874267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:22.881469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:22.887103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:22.888595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:22.904843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:22.905927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:22.906360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:22.906783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:22.923093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:22.927065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:22.928172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:22.928172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:22.928172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:22.928172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:22.928456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:22.928456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:22.930073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 396/791] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 364 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:22.937499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:22.947291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:22.948386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:22.949523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:22.951680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 396/792] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/users/swift => generated 83 bytes in 237 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:22.960795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.055896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3fe6a6-2e34-405d-a987-2412d06ebd65 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.083861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.084590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.084921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.085151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.085891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:23.086206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.087896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.101770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 397/793] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:23.107337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.170845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae247623-382d-4b78-9494-a4b60b2544a6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.201029 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:23.218124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.219065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.219637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.220907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.221996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:23.222458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.224091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b4ba81-1037-460f-96e4-e9d09c148628 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.247451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 397/794] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 11:38:22 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:23.249005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502dfa5-97db-48c3-b74e-8b61a7a67389 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.249463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502dfa5-97db-48c3-b74e-8b61a7a67389 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.249897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f502dfa5-97db-48c3-b74e-8b61a7a67389 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.255969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 398/795] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:23.256458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502dfa5-97db-48c3-b74e-8b61a7a67389 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.262630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd7df07c-f2df-41f7-ac9c-13401b371f24 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.298166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:23.298577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.299779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.300211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.300479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.309580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.311017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-73e22384-71a7-487a-8450-ff1636f44d2b None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:23.327653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:23.328647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:23.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:23.333825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 398/796] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 222 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:23.335789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.336341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.336831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.355470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:23.356781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:23.396884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f502dfa5-97db-48c3-b74e-8b61a7a67389 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.431350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 399/797] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:23.437092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.454680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.455535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.459711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.459711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.476139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.478626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.498132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:23.509283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:23.510180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:23.519492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:23.519492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:23.519492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:23.519492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:23.519492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 399/798] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 260 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:23.519492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.567602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-505c9e51-a0f7-41ff-a0d4-9a6abba8de0c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.620022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:23.638742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db12fa48-32b4-4f54-a073-0f85e35fcfb8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.680641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.681820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.682282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.682757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.698666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:23.706940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:23.708147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:23.708980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 400/799] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:23.709676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.732802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.733485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.733774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.734058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.734897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:23.735213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.736682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed89b938-e2dc-4db8-84fc-2c9c318879a0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.754080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 400/800] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 322 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:23.755397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f43543-7dee-4bae-89ca-27540fb174f2 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.755691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f43543-7dee-4bae-89ca-27540fb174f2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.756001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1f43543-7dee-4bae-89ca-27540fb174f2 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.757373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 401/801] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:23.757982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f43543-7dee-4bae-89ca-27540fb174f2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.823023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898a56ad-a1ed-4bd9-afa2-45f169737037 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.856371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:23.861225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1f43543-7dee-4bae-89ca-27540fb174f2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:23.862633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.863838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.864157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.868067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.868067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:23.868067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.868067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.884620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 401/802] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:23.884620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:23.889047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:23.896982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:23.897650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.897929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.899687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.899687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:23.899687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:23.900644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cfb94b9-fef2-4e70-bf11-272d6df5fcfe None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:23.913430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 402/803] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:23.913430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:23.913430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:23.913430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:23.930204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:23.931754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:23.992979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1e3483-8b9a-48e4-9f98-b73c12c10f74 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.007890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 403/804] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:23 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:24.007890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.039488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.040576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.040867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.041175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.052122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.053594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:24.071830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:24.072740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:24.073601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:24.073601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:24.073601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:24.073601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:24.073601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 402/805] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:24.076956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.106444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.115515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd83ddf-fc2c-4e83-a456-c632aa290a6d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.138406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.154153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.155178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.155614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.156076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.163709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.165377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:24.180194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:24.181080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:24.181943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 404/806] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:24.182709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.185479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8d0aa51-cd61-48bf-a0f3-76543587df56 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.221784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.223222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.223510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.223789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.233957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.235438 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:24.253636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:24.254550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:24.255458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:24.255458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:24.255458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:24.255458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:24.255458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 403/807] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:24.255734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.301282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ee3e98-9f35-4bf9-9134-ca5def1e2cda None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.335884 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.336707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.337246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.337529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.338186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.339553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:24.352187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:24.353144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:24.354025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 405/808] 10.222.0.40 () {56 vars in 1129 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:24.354885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.371103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff396d4b-abbf-4703-9fcf-3146dddccbac None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.374997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.382082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.408103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.409564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.410053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.410546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.411347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:24.411741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.413105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.425726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 404/809] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:24.426662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.462933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a649db9-c3a2-4a35-9941-9a53e7a4f07a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.499031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.499699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.500040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.500323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.501135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:24.501444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.502923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.517637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 406/810] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:24.518403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.543327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3642e6e-04ad-4c6a-8d65-14af2e397dd4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.576611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.577276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.577570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.577850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.578578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:24.578897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.580296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.593501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 405/811] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:24.594295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.596932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.620504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee435607-fe72-4e2c-ad00-b19c98ff3434 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.622690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.654366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.655029 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.655320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.655600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.656446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:24.656758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.658120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.662495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 407/812] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:24.663235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.697386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40cbca9a-b52b-4bd4-862c-c6697e7dca40 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.731179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.732026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.732314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.732598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.739719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.741231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-3b928d0f-5125-4315-880f-3c47fc1417e2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:24.755498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:24.756395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:24.757260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:24.761476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 406/813] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:24.762797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7261482-a791-4c7f-b057-1fed2a5aea8f None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.763092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7261482-a791-4c7f-b057-1fed2a5aea8f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.763374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7261482-a791-4c7f-b057-1fed2a5aea8f None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.763976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6621930-b026-49a6-9d13-89f60b2b7cf1 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.764678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 407/814] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:24.765379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7261482-a791-4c7f-b057-1fed2a5aea8f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.797811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.798622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.798905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.799178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/7740886489ae4fe78a1d81cd12852750/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.801780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=7740886489ae4fe78a1d81cd12852750, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.803220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.852474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.859906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:24.864769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 408/815] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:24 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/7740886489ae4fe78a1d81cd12852750/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:24.865507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.865971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7261482-a791-4c7f-b057-1fed2a5aea8f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:24.907299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:24.908251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:24.908543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:24.908840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:24.920079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:24.921659 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:24.938020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:24.939073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:24.940416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 408/816] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/users/nova => generated 82 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:24.945503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:24.984562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e31708d-2a13-4758-8f98-b6dcfaf19b16 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.021758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.023239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.023239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.023239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.023671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:25.024014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.025864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9327efe-4b9b-4ee2-952a-8bc0af172171 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.031518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 409/817] 10.222.0.40 () {56 vars in 1124 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:25.032814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.033111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.033479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.051453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:25.052757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:25.062269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5452a0-2420-4716-9836-b1af90f5c2c0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.096754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.098035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.098394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.098743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.099616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:25.099999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.101596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.115689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 409/818] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:25.116421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:25.121080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:25.125103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 410/819] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:25 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:25.125969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:25.140241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:25.233835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51bf192-0f5c-4faa-849d-a405c25fb973 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.239601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69539a3a-803c-4578-b96f-c3d13e77ceb5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.271207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.272108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.272514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.272796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.273533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.273820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.274498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.274790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.274883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.275075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.275940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:25.276260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.277712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.296677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 410/820] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:38:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:25.297478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:25.367315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:25.369264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 411/821] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:25 2022] POST /identity/v3/endpoints => generated 346 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:25.370003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:25.428574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03e00e69-5efa-4f2d-aa56-d4b09cdffa3b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.451122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:25.476619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.480192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.480798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.481286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/2e7985c45e3e478cac56a7d939e3fe2a/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.503269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=2e7985c45e3e478cac56a7d939e3fe2a, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.506183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.516041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c65169b-ba23-4472-9bee-3c347f4cfa6d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.555710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.556589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.556871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.557141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] PATH_INFO: `/v3/services/network` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.557792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.559164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:25.572272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:25.573260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:25.574539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 412/822] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:25 2022] GET /identity/v3/services/network => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:25.575358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:25.595377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 411/823] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:25 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/2e7985c45e3e478cac56a7d939e3fe2a/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:25.604204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:25.736396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db9dacc4-e3e5-48f7-93ee-ebc2c8866203 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.751956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:25.777950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.779900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.782474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.782942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.831608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a20634ab-dbe5-47e9-be90-1d23b031ddc2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:25.849899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:25.885313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:25.886538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:25.886978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:25.887790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:25.888990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] RBAC: Adding query filter params (name=network) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:25.889477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.892747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.896784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:25.898517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 412/824] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:25 2022] GET /identity/v3/services?name=network => generated 124 bytes in 302 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:25.898679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:25.903556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:26.050598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 413/825] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:25 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 476 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:38:26.056290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:26.077596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:26.082431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c332bba-156e-41d7-95bd-ff1e73131c70 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:26.151411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90827334-062b-4277-9455-69a47cbe857f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:26.152518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90827334-062b-4277-9455-69a47cbe857f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:26.153146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90827334-062b-4277-9455-69a47cbe857f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:26.154424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90827334-062b-4277-9455-69a47cbe857f None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:26.155465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90827334-062b-4277-9455-69a47cbe857f None admin] RBAC: Adding query filter params (type=network) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:26.156106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90827334-062b-4277-9455-69a47cbe857f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:26.158617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90827334-062b-4277-9455-69a47cbe857f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:26.166645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 413/826] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:25 2022] GET /identity/v3/services?type=network => generated 350 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:26.171890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90827334-062b-4277-9455-69a47cbe857f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:26.226437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b1136a-40bd-4715-a431-3eef9dd2a581 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:26.256216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:26.269616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:26.270483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:26.270794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:26.271058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:26.294795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:26.296350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:26.332355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:26.333238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:26.334713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:26.334713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:26.334713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:26.334713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:26.334713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 414/827] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:26.339499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:26.377436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90827334-062b-4277-9455-69a47cbe857f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:26.452122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:26.452879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:26.453331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:26.453619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] PATH_INFO: `/v3/endpoints` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:26.454380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:26.455758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:26.477422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:26.515284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fc79978-b9f8-4fc0-acca-01c7911511f3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:26.551897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 414/828] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:26 2022] POST /identity/v3/endpoints => generated 327 bytes in 380 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:26.576331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:26.577031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:26.577327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:26.577608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:26.578477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:26.578898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:26.580371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:26.595528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 415/829] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:26.601144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:26.744294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:26.751796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb155f14-e1cf-4601-a04c-ccb08431ccbb None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:26.789371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:26.790237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:26.790528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:26.790807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:26.797897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:26.799492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:26.814543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:26.815436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:26.816836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 415/830] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:26.824985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:26.935483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a85cfb1-49f6-4992-a592-1219c8d95304 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:26.959594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:26.973372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:26.978044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:26.978044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:26.978044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:26.978044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:26.978044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:26.978044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f228388-4839-4f36-a0ba-5cdfb76a64a6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:26.992034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 416/831] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:26.997740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:27.120275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b55ad-073d-4191-ad6c-c3c63217c895 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:27.166380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:27.167484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.168059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.168532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.176745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=54986964e91b4af7997e09d4682cc31e) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:27.176745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:27.176745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a790638-e4ec-4b57-ba9e-35ce7fc3aaed None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:27.178902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 416/832] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:26 2022] GET /identity/v3/role_assignments?user.id=54986964e91b4af7997e09d4682cc31e&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:27.183124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91982b3f-c8a2-4ccb-9383-d7cae64d01ab None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.183124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91982b3f-c8a2-4ccb-9383-d7cae64d01ab None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.183124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91982b3f-c8a2-4ccb-9383-d7cae64d01ab None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.184099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 417/833] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:27.189889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.190080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.190361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.209471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:27.210713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:27.228021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:27.288199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 417/834] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:27.312183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91982b3f-c8a2-4ccb-9383-d7cae64d01ab None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:27.427024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91982b3f-c8a2-4ccb-9383-d7cae64d01ab None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:27.461796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:27.463233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.463728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.464254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.476239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:27.484155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:27.485091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:27.485922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:27.489161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 418/835] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:27 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:27.495372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:27.596710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a509a1-d72b-4f66-9073-3bffe1c92daa None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:27.632112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:27.632784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.633069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.633344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.634097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:27.634402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:27.635754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2fe603-3bcf-4829-a837-613377966c64 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:27.645071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 418/836] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:27 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:27.650562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:27.705570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:27.745892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9b6f-1a22-4001-87f6-6571d0005b99 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.746182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9b6f-1a22-4001-87f6-6571d0005b99 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.746376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cda9b6f-1a22-4001-87f6-6571d0005b99 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.747894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 419/837] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:27 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:27.754743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.755030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.755308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.769949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5867b928-492d-4a80-b478-5ca3dc80d96c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:27.773350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:27.774547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:27.782898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:27.806293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:27.807400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:27.807928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:27.808401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:27.819065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:27.820789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:27.839491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:27.840440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:27.841294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:27.841294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:27.841294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:27.841294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:27.843465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 419/838] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:27.853005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:27.853471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 420/839] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:27 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:27.869366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:27.971195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:27.978517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f293d43d-dbf7-41cf-a0bc-26ab4ebbb5b2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:27.994139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30728f0b-2908-433d-b5f7-6af06f908528 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.014353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.014944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.015225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.015500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.016423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:28.017212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.019262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.032932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 420/840] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:28.034234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.035076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.035360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.035637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.036860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.037855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.038249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:28.050874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:28.051938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:28.052767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:28.060802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 421/841] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:27 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:28.062458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.073198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.155267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c23b5a-e9e2-4f4f-91a8-5c33503a1d1d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.168309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4100f91d-11c7-4cee-9b90-b0298e27234c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.189256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.190057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.190339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.190644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.198650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.200061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.205092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.205761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.206073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.206339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.207571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:28.207900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.209270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1628728a-57ac-4f7c-90f2-dfae18f4155f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.218725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 422/842] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:28.218867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:28.219718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jun 16 11:38:28.220689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:28.221209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc788f-f692-46a2-af1b-3b6123af8ea4 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.221209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc788f-f692-46a2-af1b-3b6123af8ea4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.221209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4bc788f-f692-46a2-af1b-3b6123af8ea4 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.221406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 421/843] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/users/swift => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:28.221406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.221993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 423/844] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:28 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:28.223916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc788f-f692-46a2-af1b-3b6123af8ea4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.240619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.304160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.325160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d847c34e-cb96-4af5-9638-2f3cb772df18 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.343584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4bc788f-f692-46a2-af1b-3b6123af8ea4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.366147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.366815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.367132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.367376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.368122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:28.368431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.369830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411636f1-9957-441f-8c98-c33497bc7c35 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.374433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 422/845] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:28.376446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.376958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.377431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.389063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.390210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.390722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.391179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.392316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:28.392822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.394682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.396025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:28.397659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:28.417012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 424/846] 10.222.0.40 () {56 vars in 1135 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:28.423575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.491894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 423/847] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:28 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:28.514129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.524865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.563509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b80d4f-135e-4b5c-8831-741e00032fb8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.601294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.607444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.608051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.608343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.608625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.623196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22ddb0f-509d-4779-bc10-7483ee022b95 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.665398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.666594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.666885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.667160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.682373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:28.688141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.689772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:28.692461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:28.693575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:28.694499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 424/848] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:28.699629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.771273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.807483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cca3f0ce-a166-4b84-836e-f29162a3c8a4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.817808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 425/849] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 394 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 11:38:28.818763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.843389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.844114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.844404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.844683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.845444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:28.845754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.847132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:28.849379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:28.861007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 425/850] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:28.861966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:28.930329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a74b4215-0af6-4f36-baa9-e63591b8fb31 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.975317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:28.976284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:28.976641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:28.976791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88286b41-2fb8-46b0-871e-f2ce994f2838 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:28.977048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:28.991425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:28.993747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.007121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.013338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.014238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.014596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.014947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:29.017357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:29.022919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:29.024359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:29.024359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:29.024359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:29.024359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:29.024359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 426/851] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:29.024993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.027833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.029435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:29.053701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:29.054629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:29.055500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:29.055500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:29.055500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:29.055500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:29.055500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 426/852] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:29.061982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.141583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.163478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2579013e-c931-42e1-99b8-7ead1e40930f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.186704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acf4c201-3514-48bb-8069-4715626bfea2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.213801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.214969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.215817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.216297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.217479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:29.218184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.220409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.228965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.229835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.230833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.231270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.232464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:29.232927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.238142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 427/853] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:29.238307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.240546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.255320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 427/854] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:29.256531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.306445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.375256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e92223a3-a9c4-46cb-861a-6736a9774721 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.414657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.415395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.415681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.415987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.423161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.424705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-562658b7-b083-4141-9d22-fcb42ddc5df6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:29.439493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:29.440460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 428/855] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e593-b60b-45cd-beae-d14e4a26eea3 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.446577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e593-b60b-45cd-beae-d14e4a26eea3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.447278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c9e593-b60b-45cd-beae-d14e4a26eea3 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.447278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 429/856] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:29.447278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e593-b60b-45cd-beae-d14e4a26eea3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.488389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca87e024-9367-44e1-8fb3-ed7157cc5fd4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.527851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.528702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.528983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.529256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.536866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.538311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.541521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-c619dd0f-1fca-499c-bf3c-cc630ba4c82a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:29.553216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:29.567658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 428/857] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.568590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.569178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.569257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88c9e593-b60b-45cd-beae-d14e4a26eea3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.569257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.573024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:29.574241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:29.603405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.604388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.604844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.605124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.605973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:29.606538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.608311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.632847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 430/858] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:29.632847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.667965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 429/859] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:29 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:29.667965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.731760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-004e5c3f-aa34-42cb-bf02-407b8233a031 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.767669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.768353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.768634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.768908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.769712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:29.770151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.771573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.779130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2300a1b-7eff-489a-9029-0f45056dc531 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.783107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 431/860] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:29.783909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.810281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.811328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:29.817845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.818542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.820037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.820037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.888154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.889593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.892280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fb4092-35d6-4a20-a7f1-57ba5812173d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:29.928085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:29.928968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:29.929253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:29.929527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:29.930294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:29.931648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:29.944648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:29.945643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:29.946455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 432/861] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:29.948055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:29.991123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 430/862] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:38:29.991123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.024618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.040779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.058848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d190fe9-9538-4f66-9c8a-357780cd61df None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.093586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066587-a479-439e-91bf-eaec48779034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.094232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066587-a479-439e-91bf-eaec48779034 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.094516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066587-a479-439e-91bf-eaec48779034 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.094791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99066587-a479-439e-91bf-eaec48779034 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.095617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066587-a479-439e-91bf-eaec48779034 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=7740886489ae4fe78a1d81cd12852750) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:30.095958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066587-a479-439e-91bf-eaec48779034 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.097935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99066587-a479-439e-91bf-eaec48779034 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.103732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 433/863] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/role_assignments?user.id=7740886489ae4fe78a1d81cd12852750&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:30.104538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066587-a479-439e-91bf-eaec48779034 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.106013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faf687ec-26c1-4dfd-8305-80d26782bcbb None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.159552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.160915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.161202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.161474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.162796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:30.163116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.164934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.173226 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 431/864] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:29 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:30.177495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.229978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99066587-a479-439e-91bf-eaec48779034 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.274834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.275767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.276163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.276463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.289004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.290467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.301536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:30.311154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:30.312163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:30.313021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:30.313021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:30.313021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:30.313021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:30.313247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 434/865] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:30.319422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.320040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.327805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-520fed98-9aab-4ce0-a98e-a48d75a4e425 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.374175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.375078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.375257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.375502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.376232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:30.376539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.380330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.384638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 432/866] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:30 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:30.443405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9185b15-6dc0-4042-9609-1d537feba2d0 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.487541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.488674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.489163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.489644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.490593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:30.491105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.492747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-634ae9c2-75d4-455a-a61b-c93dcfa66226 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.507218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 435/867] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:30.512891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.594570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e5f0e-1029-4a51-a7ea-a3be0fddd860 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.594889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e5f0e-1029-4a51-a7ea-a3be0fddd860 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.595137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442e5f0e-1029-4a51-a7ea-a3be0fddd860 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.596675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 436/868] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:30.602578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.602991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.603372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.621353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:30.622909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:30.625779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-980f2fc6-8634-4689-9f00-cce2da2178a2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.663776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.664835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.665303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.665744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.675129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.676995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:30.692796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:30.693687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:30.694984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 433/869] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:30.699833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.700820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.713806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 437/870] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:30 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:30.724753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.736264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.808086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdf61e99-fb55-43d5-897b-95d0961b700a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.843147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.843943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.844234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.844507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.845293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:30.845600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.847250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.860415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 434/871] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:30.865846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5308aeee-ff8f-4752-8754-d7c23720e75b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:30.866338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:30.911336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae950a98-3be8-427c-9ddf-c20e2f69f281 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:30.912050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae950a98-3be8-427c-9ddf-c20e2f69f281 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:30.912393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae950a98-3be8-427c-9ddf-c20e2f69f281 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:30.912669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae950a98-3be8-427c-9ddf-c20e2f69f281 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:30.913454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae950a98-3be8-427c-9ddf-c20e2f69f281 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:30.914825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae950a98-3be8-427c-9ddf-c20e2f69f281 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:30.959323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:30.982118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 438/872] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:30 2022] POST /identity/v3/services => generated 251 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:30.983314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ac6f5f6-3d20-46c5-91d9-34b199ba3ba1 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:31.035475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:31.036237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:31.036529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:31.036802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:31.037798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=2e7985c45e3e478cac56a7d939e3fe2a) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:31.038116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:31.039604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a716fe-6199-4a23-bdaf-20ad20ca2573 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:31.047086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 435/873] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:30 2022] GET /identity/v3/role_assignments?user.id=2e7985c45e3e478cac56a7d939e3fe2a&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:32.399808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad818c3d-5dc5-4604-9013-d63fd0fb692e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.400706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad818c3d-5dc5-4604-9013-d63fd0fb692e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.401236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad818c3d-5dc5-4604-9013-d63fd0fb692e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.402965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 439/874] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:32.408416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031625d-c6d9-452f-9289-b7b7ec270179 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.409237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031625d-c6d9-452f-9289-b7b7ec270179 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.410289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9031625d-c6d9-452f-9289-b7b7ec270179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.427051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9031625d-c6d9-452f-9289-b7b7ec270179 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:32.428586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9031625d-c6d9-452f-9289-b7b7ec270179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:32.498015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 436/875] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:32.520820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad818c3d-5dc5-4604-9013-d63fd0fb692e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:32.582298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:32.612776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d4c6c-19f4-44ea-8e84-8ef23fbbb751 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.613436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d4c6c-19f4-44ea-8e84-8ef23fbbb751 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.613901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e12d4c6c-19f4-44ea-8e84-8ef23fbbb751 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.615738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 437/876] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:32.621579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.622355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.622879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.634958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad818c3d-5dc5-4604-9013-d63fd0fb692e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:32.646945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:32.648479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:32.671644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:32.672421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.672710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.672992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.673657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:32.675158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:32.690647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:32.692170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:32.692982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:32.696019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 440/877] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:32 2022] GET /identity/v3/services/object-store => generated 93 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:32.698638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:32.730360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:32.744015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 438/878] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:32 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:32.765420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:32.811801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5008a659-a81f-4a06-b85a-4fc8d322a66b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:32.829011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:32.850425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:32.851479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.852003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.852486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.853418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:32.853950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:32.855516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:32.860932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 441/879] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:32 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:32.866573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:32.890750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe769097-80ff-4d37-bb96-9bff9b1deaf9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:32.930060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d0db36-b163-4298-9447-65d2dece624f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:32.931124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0db36-b163-4298-9447-65d2dece624f None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:32.931614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0db36-b163-4298-9447-65d2dece624f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:32.932127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d0db36-b163-4298-9447-65d2dece624f None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:32.933207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0db36-b163-4298-9447-65d2dece624f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:32.935654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d0db36-b163-4298-9447-65d2dece624f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:32.979931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e333e90-8bfc-4f61-8c31-6047eaec0113 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:33.015353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:33.016101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.016387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.016686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.017448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:33.017756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:33.019136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cfa529-5190-4a3e-885d-9ac30e9bc31f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:33.023808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 442/880] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:32 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:33.038327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 439/881] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:32 2022] POST /identity/v3/services => generated 269 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:33.210580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0232-4112-4de9-9566-61649dd91e77 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.210978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0232-4112-4de9-9566-61649dd91e77 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.211302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043d0232-4112-4de9-9566-61649dd91e77 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.213008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 443/882] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:33.219507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.219799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.220096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.240674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:33.242280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:33.298600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c5c59-4da3-4661-bf76-c2f80d76f95a None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.298887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c5c59-4da3-4661-bf76-c2f80d76f95a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.299169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0c5c59-4da3-4661-bf76-c2f80d76f95a None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.302368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 444/883] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:33.302503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:33.307018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.307357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.307583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.326975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 440/884] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:33.331644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:33.331644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:33.356896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.373993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:33.427927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 445/885] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:33 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:33.451251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.485387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c5315f6-aa7b-41a6-ab05-99b928db7cf6 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:33.528721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:33.529639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.529935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.530217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.530933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:33.532307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:33.547374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:33.548428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:33.549238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:33.552409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 441/886] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:33.555301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.586891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be16a31-949b-4194-b02a-13f4e3ca2191 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:33.615571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:33.637222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:33.638847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.639135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.639416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.652792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:33.663619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:33.664595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:33.665428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 446/887] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:33.670571 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.675095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22a1661d-b8d1-4dea-b00d-627a5e0148bd None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:33.710573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:33.711293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.711583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.712333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.713087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:33.713492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:33.714864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:33.717337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:33.726467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 442/888] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:33.732847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.803802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f01e81-8c41-4680-9015-f9ff71ef0bce None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:33.846917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:33.848510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.848852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.849126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.850578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:33.850936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:33.852877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:33.859486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-433dd5ad-3e99-4951-8b31-7dd26d844dd0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:33.864689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 447/889] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:33.870912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.879075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:33.901662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:33.902442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:33.902795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:33.903140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:33.903984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:33.904361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:33.905798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:33.912327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 443/890] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:33.918697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:33.989789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:33.996302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1668ded0-1ccb-4cf9-8ef3-2c61c552cc6c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.031759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.032762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.033154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.033561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.043989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.045530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.056518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53f3171c-c93e-4964-bd4f-6de2ae1d3269 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:34.071116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:34.072422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:34.073638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:34.073638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:34.073638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:34.073638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:34.079612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 448/891] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:34.080600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:34.100344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.101133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.101422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.101765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.102817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] RBAC: Adding query filter params (interface=public, service_id=455822c9f34544dd9b90b1f36fa234aa, region_id=RegionOne) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:34.103130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.104873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.113702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 444/892] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:33 2022] GET /identity/v3/endpoints?service_id=455822c9f34544dd9b90b1f36fa234aa&interface=public®ion_id=RegionOne => generated 194 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:34.119000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:34.173616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:34.207963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a557de-2e92-4ae0-8529-e2ea0f13a339 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.238422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbaf66a9-9c9b-4951-95f7-1d64d2bf8464 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.257368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.258050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.258481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.258763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.260023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:34.260337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.261707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.278876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.279549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.280035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.280332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.281043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.281833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 449/893] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:34.282743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:34.282864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.290217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 445/894] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:34 2022] GET /identity/v3/services => generated 1268 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:38:34.295048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:34.400220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cf064-6156-48a2-ad3f-7d0195da4651 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.438361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.439459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.439740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.440045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.449369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.452091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:34.467698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:34.469882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:34.472256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 450/895] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:38:34 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:34.475190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:34.577022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3196b9a-7234-4f5a-82f6-a44dc2b5c764 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.579338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:34.610334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.611064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.611347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.611623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.612534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:34.613376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.614781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.638857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:34.639275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 446/896] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 11:38:34 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:34.710623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:34.746506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a68289-e7ce-4662-8b9c-faab533f8332 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:34.781894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:34.782547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:34.782828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:34.783100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:34.855207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:34.856675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:34.923099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:34.958988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 451/897] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:34 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 320 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:38:34.964794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.066487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e135837-25f8-474e-b4c1-c25b2513306a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.101691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.102539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.102823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.103098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.114196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.115649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5c1e4de5-ed2b-444e-8bd2-2f8523adbf0a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:35.133567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:35.134593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:35.135480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:35.135480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:35.135480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:35.135480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:35.135480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 447/898] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:35.140664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.248141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d55a0432-eec4-40fb-a754-dd7db41c9625 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.270658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9780967-3870-4a7c-9ddc-078e785b2789 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.270969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9780967-3870-4a7c-9ddc-078e785b2789 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.271205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9780967-3870-4a7c-9ddc-078e785b2789 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.273863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 448/899] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:35 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:35.277941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8259403-8b1f-4b69-87b9-ffcc80d1975d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.278232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8259403-8b1f-4b69-87b9-ffcc80d1975d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.278513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8259403-8b1f-4b69-87b9-ffcc80d1975d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.287090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.287762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.288071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.288350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.289074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:35.289379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.291160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4ad7ed-ecb2-44f3-9a10-89b0f8938f0a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.297029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c8259403-8b1f-4b69-87b9-ffcc80d1975d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:35.298321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c8259403-8b1f-4b69-87b9-ffcc80d1975d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:35.304383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 452/900] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:35.306348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6dc7c-4f3f-4e59-a1f3-339aacc7676a None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.306640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6dc7c-4f3f-4e59-a1f3-339aacc7676a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.306921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34b6dc7c-4f3f-4e59-a1f3-339aacc7676a None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.308258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 453/901] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:35.309474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6dc7c-4f3f-4e59-a1f3-339aacc7676a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.312436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:35.340358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:35.373338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 449/902] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:35.381633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.381930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.382207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.406518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:35.407703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:35.423720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34b6dc7c-4f3f-4e59-a1f3-339aacc7676a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.458710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.459506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.459789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.460088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.467613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.469167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:35.484770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:35.485712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:35.486563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 454/903] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:35.487049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.493177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 450/904] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:35 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:35.496753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.580993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:35.582988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:35.597077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f7829f2-d07b-4a78-afa4-68cedcd74c81 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.610339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06c68892-d698-458d-ae4d-72f4e6526260 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.630337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.631001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.631281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.631555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.632310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.633665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.651011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.651685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.652012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.652295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.653084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:35.653398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.654817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.666072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 451/905] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:35.666844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.713142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 455/906] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:35 2022] POST /identity/v3/services => generated 241 bytes in 226 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:35.714762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.793396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-979c449c-9dba-45eb-8635-163b7eb67623 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.861994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:35.869188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.870875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.871550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.872282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.873507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.875776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.887679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7364e5b4-3347-422e-b38c-bc77aaf94be2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:35.891910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:35.892838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:35.893665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:35.896676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 452/907] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jun 16 11:38:35 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 230 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:35.904895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:35.944138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:35.944779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:35.945105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:35.945381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:35.946214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] RBAC: Adding query filter params (role.id=405b058de2424949a5439d54038e3e48, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=2e7985c45e3e478cac56a7d939e3fe2a) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:35.946552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:35.949576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:35.963979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 456/908] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:35 2022] GET /identity/v3/role_assignments?user.id=2e7985c45e3e478cac56a7d939e3fe2a&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=405b058de2424949a5439d54038e3e48 => generated 259 bytes in 249 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:36.030129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c20fff2-8bc7-4fa3-87b1-2505e818a561 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:36.067847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:36.068929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.069411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.069894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.070848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:36.071389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:36.073016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:36.079243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 453/909] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:35 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:36.087928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:36.110462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:36.254209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec67c932-1bbb-4959-a3bd-26ed54cacacd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:36.311313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:36.312390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.312894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.313380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.314303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:36.314805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:36.316426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:36.322163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 457/910] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:36 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 238 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:36.327623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:36.388483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:36.434319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd6fd69b-898d-4852-8f41-579a1aa9b067 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:36.471325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:36.472898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.473403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.473893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.474890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] RBAC: Adding query filter params (interface=public, service_id=1daec7085eaf4b288844089efa6682a2, region_id=RegionOne) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:36.475403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:36.477026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1507e12-fbae-442a-b52e-cae92c1c2de6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:36.482981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 454/911] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:36 2022] GET /identity/v3/endpoints?service_id=1daec7085eaf4b288844089efa6682a2&interface=public®ion_id=RegionOne => generated 194 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:36.489307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:36.491506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce085d4-2149-4a86-baf3-b24ebdb39ef9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.491800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce085d4-2149-4a86-baf3-b24ebdb39ef9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.492118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ce085d4-2149-4a86-baf3-b24ebdb39ef9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.493714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 455/912] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:36.498590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.498875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.499154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.517857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:36.519604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:36.571505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:36.598947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80ad6b79-7c1b-44ac-a3a4-2c3bbf38b569 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:36.608011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 456/913] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:36 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:36.630990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:36.645262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:36.645942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.646231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.646509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.647225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:36.649177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:36.663254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 458/914] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:36 2022] GET /identity/v3/services => generated 1497 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:38:36.724429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:36.771228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73f1fed0-3ce9-4978-84f6-8ff0d6d62aad None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:36.809584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:36.810658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.811119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.811558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.812417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:36.813973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:36.827259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:36.828445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jun 16 11:38:36.829278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:36.830011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 457/915] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:36 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:36.836673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:36.942896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367dca8e-7192-4a76-beb6-de6b817794ce None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:36.978249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:36.978915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:36.979201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:36.979480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:36.980217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:36.980528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:36.981954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:36.987216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 459/916] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:36 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:36.992768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:37.100082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f12a83-2878-4a83-9529-feda1315e462 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:37.122242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:37.137069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:37.137809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:37.138095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:37.138373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:37.139135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:37.139447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:37.140840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ee4ccfb-3528-40de-b333-35fe0dd17d64 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:37.146694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 458/917] 10.222.0.40 () {56 vars in 1114 bytes} [Thu Jun 16 11:38:36 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:37.152335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:37.156393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:37.257445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b742d46-5b12-4042-9e05-ced077207424 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:37.293451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c36f2884-3b62-4e62-ae95-5529738b128c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:37.294433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f2884-3b62-4e62-ae95-5529738b128c None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:37.294922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f2884-3b62-4e62-ae95-5529738b128c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:37.295424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c36f2884-3b62-4e62-ae95-5529738b128c None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:37.296403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f2884-3b62-4e62-ae95-5529738b128c None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:37.298083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c36f2884-3b62-4e62-ae95-5529738b128c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:37.368672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 460/918] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:37 2022] POST /identity/v3/endpoints => generated 346 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:38.082676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af936b0-5f89-4419-9b85-75de90b5710c None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.083227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af936b0-5f89-4419-9b85-75de90b5710c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.083227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af936b0-5f89-4419-9b85-75de90b5710c None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.084924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 459/919] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:38.090950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.091481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.091481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.110444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:38.111923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:38.163610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cef70-b030-45b8-9957-63934f6d52c6 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.163946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cef70-b030-45b8-9957-63934f6d52c6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.164215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812cef70-b030-45b8-9957-63934f6d52c6 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.165836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 460/920] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:38.170716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.171007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.171287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.171769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:38.187719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:38.188961 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:38.199381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 461/921] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:38.226204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.262114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 461/922] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:38 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:38.286408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.348962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fca60fe-1420-40ac-b5b6-7c95ef62c4a6 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.387356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60cb5e7-6647-4d59-9040-709ad9d14518 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.391739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.392743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.393196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.393613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.394935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:38.396780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:38.404215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:38.409404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:38.410904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:38.411728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:38.416307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 462/923] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/services/object-store => generated 93 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:38.417200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.433153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.435115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.435600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.436098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.449434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:38.460518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:38.461527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:38.462618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:38.465755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 462/924] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:38.471504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.478645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:38.536976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1b65e03-a594-4164-907d-85c700ea245c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.573311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.574043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.574331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.574655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.575485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:38.575911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:38.577280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:38.583669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 463/925] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:38.590432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.598879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-763aa086-ebfd-4072-9ca8-1efeda2be95a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.638128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.639216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.639701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.640238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.641307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:38.641817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:38.643554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:38.658151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 463/926] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:38.659395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.697050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:38.708029 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22ac64d7-9f9f-4cc2-94f0-8d23f29b26bd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.722017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:38.745140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.746141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.746653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.747238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.748225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:38.748700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:38.750342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:38.762435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 464/927] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:38.763581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.781365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2f0f56-47d9-445a-8c48-eb14fcfe8361 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.830191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.831932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.832224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.832502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.848577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:38.850032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:38.871367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:38.872366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:38.873261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:38.873261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:38.873261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:38.873261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:38.876225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 464/928] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:38.881267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.887574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5bc1306-3a86-4b36-8ec9-79631605df7b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.928491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:38.929446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.929882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.930163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.930975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] RBAC: Adding query filter params (interface=public, service_id=30671ed63def45409e672e328a8d23c8, region_id=RegionOne) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:38.931295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:38.932700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8311e75f-d4f1-484f-a5e4-810b9c4bfcfa None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:38.942046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 465/929] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/endpoints?service_id=30671ed63def45409e672e328a8d23c8&interface=public®ion_id=RegionOne => generated 194 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:38.942046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae623d-961d-4141-9a5b-8de3d0dae4f0 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:38.942046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae623d-961d-4141-9a5b-8de3d0dae4f0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:38.942046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aae623d-961d-4141-9a5b-8de3d0dae4f0 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:38.942046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 466/930] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:38.946572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae623d-961d-4141-9a5b-8de3d0dae4f0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:38.972833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:38.991081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce835e26-8ded-4379-9eac-2cc829c18dca None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:38.992182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.031707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.032404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.032683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.032956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.033673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:39.034018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.035409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ac91fd-a3dc-4e68-8a8f-a3040b7dca18 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.047425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 465/931] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:39.048675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.048967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.049239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.054835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aae623d-961d-4141-9a5b-8de3d0dae4f0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.065488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:39.067263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:39.091395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.092119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.092408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.092690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.093377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.094745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.107909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 467/932] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:38 2022] GET /identity/v3/services => generated 1497 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:38:39.107909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.171318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 466/933] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:39.195378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.239206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1f4c3c8-907f-4d1a-9ff5-29e285b322db None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.243105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.279089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.280081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.280759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.281118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.296708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.298162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.299746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:39.315181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:39.316329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.317271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 468/934] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:39.321868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.327553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d1d282-1046-42bd-8c66-d3896e684c07 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.365338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.366157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.366438 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.366712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.367366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.368760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:39.381240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:39.382269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:39.383127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 467/935] 10.222.0.40 () {56 vars in 1132 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:39.388236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.465789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a773d49c-e462-4ea2-884a-5d4a971c7ebd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.506004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29abbaae-c885-431a-80a8-f83a78098234 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.511822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.512612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.512896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.513172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.514016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:39.514323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.517917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8b5fd-f7e0-442e-a51b-360ddc9108f1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.542395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 469/936] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:39.543860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20976b1-b138-4f8f-a0d8-6ea6b59eb032 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.544192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20976b1-b138-4f8f-a0d8-6ea6b59eb032 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.544466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c20976b1-b138-4f8f-a0d8-6ea6b59eb032 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.546873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.547030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 470/937] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:39.548013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.548373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20976b1-b138-4f8f-a0d8-6ea6b59eb032 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.548720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.549212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.550935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:39.551471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.552881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e7eb239-7960-4132-87db-dc3245d20c64 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.558656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 468/938] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:39.559752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.560093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.560379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.565603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.577429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:39.578608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:39.582106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.652229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 469/939] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:39 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:39.653107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.671231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c20976b1-b138-4f8f-a0d8-6ea6b59eb032 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.716375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.717090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.717374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.717651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.728777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.730240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:39.747519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:39.748448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:39.749328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:39.749328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:39.749328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:39.749328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.749328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 471/940] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:39.749596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.773382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d74319-5629-404d-a3ef-7ff3104f80d6 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.805986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.810855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.811538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.811825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.812133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.812847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:39.813159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.814624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.819372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 470/941] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:39.824147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.836329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:39.863004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f7635d-7cdf-4eeb-8e2e-c282c6319034 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.898721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.899428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.899665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.899969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.900673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.902096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.903503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING keystone.server.flask.common [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jun 16 11:38:39.930840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9649d557-06aa-4fa4-a6d8-1f8aa5ed1394 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:39.944021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.sql.core [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 11:38:39.944021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 11:38:39.944021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [parameters: {'id': '9238fad366864799bbf2bf0e9ffc9d35', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 11:38:39.944021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87092) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9238fad366864799bbf2bf0e9ffc9d35.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9238fad366864799bbf2bf0e9ffc9d35. Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application conn.query(q) Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 11:38:39.953956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result.read() Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jun 16 11:38:39.954783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return project_ref.to_dict() Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application next(self.gen) Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application yield resource Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application next(self.gen) Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise self.value Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application session.commit() Jun 16 11:38:39.955728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.session.flush() Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application compat.raise_( Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 11:38:39.957170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application c = connection._execute_20( Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 11:38:39.959312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application conn.query(q) Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result.read() Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 11:38:39.961100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [parameters: {'id': '9238fad366864799bbf2bf0e9ffc9d35', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.963754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:39.965568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.Conflict( Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 9238fad366864799bbf2bf0e9ffc9d35. Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 472/942] 10.222.0.40 () {60 vars in 1140 bytes} [Thu Jun 16 11:38:39 2022] POST /identity/v3/projects => generated 262 bytes in 206 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:39.968145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:39.976529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:39.977303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:39.977638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:39.977979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:39.978770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:39.979142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:39.980603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:39.997218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 471/943] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:39.997218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:40.053216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:40.063607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2951c9d-5bad-473d-b5d7-9c1a95c732ce None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:40.096348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:40.100212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915c61b5-a1e5-41e0-b9e6-4571fa7e33a7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:40.102255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:40.102925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:40.103196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:40.103468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:40.104231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:40.105809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:40.135207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:40.136057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:40.136347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:40.136623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:40.143282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:40.145413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:40.159258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:40.160325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 472/944] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:40.161440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:40.217620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 473/945] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:39 2022] POST /identity/v3/endpoints => generated 347 bytes in 262 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:40.218568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:40.295900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ecaaa5a6-7508-4c5f-8913-9ece593677ac None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:40.318251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:40.351829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:40.352866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:40.353279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:40.353574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:40.355082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61072219-b728-44d9-89c0-83330ab779ef None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:40.363200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:40.365855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:40.386146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:40.387279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:40.388411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:40.389507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 473/946] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:40 2022] GET /identity/v3/projects/service => generated 88 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:40.395591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:40.400087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:40.400964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:40.401396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:40.401819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:40.402736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:40.403204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:40.404840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:40.424730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 474/947] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:40 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:40.510744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d153b5-e4de-473a-89d5-61e27eb33f3e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:40.548316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:40.548999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:40.549284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:40.549565 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:40.550427 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:40.550738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:40.552305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d2c8c73-7c79-4a9e-b87d-42cd587dd75c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:40.566289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 474/948] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:40 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:40.572303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:40.622355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:40.682198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1410ce10-d5f7-4687-aeec-d7f67aee906b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:40.725596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1ed33c7-c848-449d-ad5a-053cfab8ea10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:40.726904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed33c7-c848-449d-ad5a-053cfab8ea10 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:40.727392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed33c7-c848-449d-ad5a-053cfab8ea10 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:40.727895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1ed33c7-c848-449d-ad5a-053cfab8ea10 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/2e7985c45e3e478cac56a7d939e3fe2a/roles/405b058de2424949a5439d54038e3e48` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:40.730377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed33c7-c848-449d-ad5a-053cfab8ea10 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=2e7985c45e3e478cac56a7d939e3fe2a, role_id=405b058de2424949a5439d54038e3e48)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:40.732123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1ed33c7-c848-449d-ad5a-053cfab8ea10 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:40.840598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 475/949] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:38:40 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/2e7985c45e3e478cac56a7d939e3fe2a/roles/405b058de2424949a5439d54038e3e48 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:38:41.298614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bd59c-430d-43a5-82bc-b2eb88daacb9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:41.299139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bd59c-430d-43a5-82bc-b2eb88daacb9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:41.299385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-535bd59c-430d-43a5-82bc-b2eb88daacb9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:41.301475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 475/950] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:41.307137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:41.307992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:41.308489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:41.326386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:41.328281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:41.398614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 476/951] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:41 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:41.421738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bd59c-430d-43a5-82bc-b2eb88daacb9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:41.455612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:41.531551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-535bd59c-430d-43a5-82bc-b2eb88daacb9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:41.571976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:41.572706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:41.572986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:41.573258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:41.573908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:41.575269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:41.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:41.589380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:41.590256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 476/952] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:41 2022] GET /identity/v3/services/object-store => generated 93 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:41.597798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:41.641073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:41.718956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd15b23a-a4b8-49b9-8f85-b72bbbead205 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:41.755262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:41.756209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:41.756637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:41.757057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:41.757903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:41.758353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:41.759886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:41.765383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 477/953] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:41 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:41.771713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:41.867373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:41.873950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafa871f-96c7-47d9-9502-25b05f8d63d7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:41.910720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:41.911617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:41.912073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:41.912488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:41.913358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:41.913831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:41.915327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1abef6f7-4552-46c8-bd6f-4a4b65315321 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:41.920828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 477/954] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:41 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:41.931900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:42.027007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:42.033591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c118f9-23da-4472-bbd6-d221885823cc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:42.069051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9acc31ab-2220-4da0-ac36-1e21f89a7c54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:42.069699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc31ab-2220-4da0-ac36-1e21f89a7c54 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:42.069984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc31ab-2220-4da0-ac36-1e21f89a7c54 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:42.070259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9acc31ab-2220-4da0-ac36-1e21f89a7c54 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:42.071004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc31ab-2220-4da0-ac36-1e21f89a7c54 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:42.072401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9acc31ab-2220-4da0-ac36-1e21f89a7c54 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:42.151070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 478/955] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:41 2022] POST /identity/v3/endpoints => generated 349 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:42.557169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e67a-9789-4811-87e4-bb3d99eea129 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:42.558165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e67a-9789-4811-87e4-bb3d99eea129 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:42.558667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf6e67a-9789-4811-87e4-bb3d99eea129 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:42.560643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 478/956] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:42.566695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e40aa-a252-4c81-8276-4b531b08cd9e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:42.567546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e40aa-a252-4c81-8276-4b531b08cd9e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:42.568061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c1e40aa-a252-4c81-8276-4b531b08cd9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:42.586025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8c1e40aa-a252-4c81-8276-4b531b08cd9e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:42.587787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8c1e40aa-a252-4c81-8276-4b531b08cd9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:42.671810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 479/957] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:42.689434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e67a-9789-4811-87e4-bb3d99eea129 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:42.766665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110acb65-8e5f-4cb5-8dd0-86decfc86917 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:42.767328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110acb65-8e5f-4cb5-8dd0-86decfc86917 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:42.767814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110acb65-8e5f-4cb5-8dd0-86decfc86917 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:42.769732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 480/958] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:42 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:42.774790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:42.775311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:42.775802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:42.792775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:42.794268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:42.801426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf6e67a-9789-4811-87e4-bb3d99eea129 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:42.838805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:42.840174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:42.840685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:42.841169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:42.842040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:42.843608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:42.856010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:42.857016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:42.857852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:42.860712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 479/959] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:42 2022] GET /identity/v3/services/compute => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:42.869168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:42.890387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 481/960] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:42 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:42.903108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:42.914439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:42.928190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.004955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57a934d-485c-48f4-81ee-92467bdfdb91 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.041478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.042137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.042421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.042710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.043396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:43.043700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6014973-0a89-426c-a430-914c3bd9d68c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.043825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.045138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8e7c71-23b9-415f-994c-d8628f17e837 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.055858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 480/961] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:42 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:43.055858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799e493-3dc8-4c3f-8437-9f62ba39f06d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.055858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799e493-3dc8-4c3f-8437-9f62ba39f06d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.055858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3799e493-3dc8-4c3f-8437-9f62ba39f06d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.055858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 481/962] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:43 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:43.057913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799e493-3dc8-4c3f-8437-9f62ba39f06d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.085438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.086109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.086391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.086860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.087607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.089114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.132238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.sql.core [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 11:38:43.132238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 11:38:43.132238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [parameters: {'user_id': '32080e74f7f54b568e5a98ad65599ec0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 11:38:43.132238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=87092) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-c25df0ee-534b-4a1d-aaf0-480ca672c826 None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application conn.query(q) Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result.read() Jun 16 11:38:43.141512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 11:38:43.148915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application next(self.gen) Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application yield resource Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application next(self.gen) Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.session.rollback() Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.force_reraise() Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise self.value Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application session.commit() Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jun 16 11:38:43.149868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._prepare_impl() Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.session.flush() Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._flush(objects) Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application compat.raise_( Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application flush_context.execute() Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rec.execute(self) Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jun 16 11:38:43.150745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application _emit_insert_statements( Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = connection._execute_20( Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 325, in _execute_on_connection Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ret = self._execute_context( Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jun 16 11:38:43.151627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self.dialect.do_execute( Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result = self._query(query) Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application conn.query(q) Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application result.read() Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jun 16 11:38:43.152678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application packet.raise_for_error() Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [parameters: {'user_id': '32080e74f7f54b568e5a98ad65599ec0', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:43.153819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jun 16 11:38:43.155527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 482/963] 10.222.0.40 () {60 vars in 1131 bytes} [Thu Jun 16 11:38:42 2022] POST /identity/v3/users => generated 160 bytes in 230 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.157437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.160352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.164399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3799e493-3dc8-4c3f-8437-9f62ba39f06d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.169872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:43.175922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:43.199189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.199853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.200163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.200550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.201302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:43.201562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.202925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.207944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 482/964] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:43.207944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.245787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.260378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 483/965] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:43 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:43.276457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.364182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87845cf7-72e2-43c2-af1b-81447ddd8a12 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.400530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-408ca4ab-292e-4f1f-b1ce-6713dc842cc4 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.411632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.412754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.412891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.413155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] PATH_INFO: `/v3/users/cinder` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.421061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.422621 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:43.438841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:43.439814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:43.440929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 483/966] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/users/cinder => generated 84 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:43.447163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.447763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.448596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.449106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.449601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.462504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:43.470948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:43.472491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.473939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 484/967] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:43.478719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.492248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.564243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.574251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8d69fd2-69c1-4d21-ae6c-bf4d0907185a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.603250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e46e59e3-d818-4cf0-918d-fa6e7316ec62 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.624371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.625587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.626062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.626672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.628037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] RBAC: Adding query filter params (name=cinder) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:43.628805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.630900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.638143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.638889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.639240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.639598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.640437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:43.640812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.642532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.655689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 485/968] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:43.661649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.674658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 484/969] 10.222.0.40 () {56 vars in 1101 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 227 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:43.782420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d32d546-9fec-4a88-af3e-57e0893bf2d3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:43.808142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.821311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:43.822139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:43.822426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:43.822706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:43.835014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:43.836599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:43.855776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:43.856730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:43.858636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:43.858636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:43.858636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:43.858636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:43.860416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 486/970] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:43.865001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:43.941679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:43.975230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eafdf56e-5543-42bc-be74-925cde201f5d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.009065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.009730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.010015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.010294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.011015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:44.011323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.012714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.025102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 485/971] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:44.030375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.130334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3267e678-3b52-468b-a1e4-a9f18453e745 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.163622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.164469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.164756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.165034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.172044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.173579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-cf6eea6e-7cb2-4f6e-85de-24165ec7ebf6 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:44.188791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:44.189741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jun 16 11:38:44.191298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:44.195971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 487/972] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:44.196240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.301979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07e1d8a-4cc0-4f95-885d-f9c849e554f9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.315065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6a0f2-c971-4bb8-88f2-60450d960c04 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.316081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6a0f2-c971-4bb8-88f2-60450d960c04 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.316375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85c6a0f2-c971-4bb8-88f2-60450d960c04 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.317873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 488/973] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:44 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:44.322560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.322848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.323125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.333279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:44.339508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:44.340025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.340726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:44.340835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.341046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.345292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.345292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:44.345292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.345292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.362122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 486/974] 10.222.0.40 () {56 vars in 1149 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:44.368505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.373458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:44.419903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 489/975] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:44 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:44.441136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.474718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f89727b-8126-4844-abd8-49f1e81d8272 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.512236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.513126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.513411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.513688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.549398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97975ad0-023b-436b-98b7-50ed2bea968d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.575672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:44.585878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.586687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.587222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.587510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.592530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.592530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.592698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.592969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:44.602486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:44.604217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:44.605701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 490/976] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/services/object-store => generated 93 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:44.609389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.639284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:44.701329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 487/977] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 11:38:44.706499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.721148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8ddd598-c1c6-4665-ae9a-ed6554d8d913 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.758854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.759451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.759741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.760043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.760757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:44.761068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.762462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.767395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 491/978] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:44.772659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.799750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:44.814231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f14d8b-6f7c-415f-a251-6431ff878999 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.857208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.858004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.858288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.858773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.869664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.870150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:44.871506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.878834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c520ef-381d-4131-b8f2-43014e05025a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:44.889852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:44.890765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:44.892474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:44.892474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:44.892474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:44.892474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:44.892474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 488/979] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/domains/Default => generated 87 bytes in 185 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:44.897164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:44.918071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:44.918752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:44.919096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:44.919378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:44.920133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:44.920447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:44.921815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:44.927648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 492/980] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:44.934098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:45.004454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4382353e-dcd1-42e0-8c91-328a4dd8c4c7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.039914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d4b260b-dff5-4000-9959-07636001857b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.046870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.048077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.048369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.048647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.049841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:45.050170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.051991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.061839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:45.069839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 489/981] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:45.070937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:45.078843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.079518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.079925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.080216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.082234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] RBAC: Adding query filter params (interface=admin, service_id=30671ed63def45409e672e328a8d23c8, region_id=RegionOne) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:45.082234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.083231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.088783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 493/982] 10.222.0.40 () {56 vars in 1249 bytes} [Thu Jun 16 11:38:44 2022] GET /identity/v3/endpoints?service_id=30671ed63def45409e672e328a8d23c8&interface=admin®ion_id=RegionOne => generated 193 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:45.094241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:45.117310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:45.178310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82d4ca9b-4374-4060-be51-d1cb362c8509 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.196353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-796e8e86-05cd-4dd7-b804-64fe2cfade22 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.217049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.217887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.218174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.218449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.226624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.228262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.233764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.234436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.234723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.235087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.235676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.237061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.243481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 494/983] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:45 2022] GET /identity/v3/services => generated 1497 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-e0d0db2d-2965-4987-8d44-1387f0ad2b35 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:45.243592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:45.244749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:45.245932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:45.248248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 490/984] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:45 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:45.250278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:45.345774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:45.364547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a71dea8-9385-49b5-9bc7-5de1b7621255 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.403893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.404567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.404853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.405129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.406256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:45.406256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.407648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.412727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4bfeb-acb6-4fa0-a276-c4480cb2f433 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.413014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4bfeb-acb6-4fa0-a276-c4480cb2f433 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.413292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b4bfeb-acb6-4fa0-a276-c4480cb2f433 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.414645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 491/985] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:45.419721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.420046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.420338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.422049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 495/986] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:45 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:45.428864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:45.439556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:45.441317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:45.501394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:45.515972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 492/987] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:45 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:45.535535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5aa7d95-684f-4098-9d13-654737a41192 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.537647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:45.572895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.573730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.574116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.574494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.575457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] RBAC: Adding query filter params (role.id=405b058de2424949a5439d54038e3e48, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=2e7985c45e3e478cac56a7d939e3fe2a) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:45.575908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.577430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8a31b44-f6dc-46ee-a4c9-a22b33bfb709 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.584445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 496/988] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:45 2022] GET /identity/v3/role_assignments?user.id=2e7985c45e3e478cac56a7d939e3fe2a&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=405b058de2424949a5439d54038e3e48 => generated 608 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:45.644879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e854a22-7673-4667-b0d1-3fb568c85e57 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:45.687901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aad51d72-bc76-4c08-95d8-fb0cb9352550 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:45.688608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad51d72-bc76-4c08-95d8-fb0cb9352550 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:45.688892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad51d72-bc76-4c08-95d8-fb0cb9352550 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:45.689171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aad51d72-bc76-4c08-95d8-fb0cb9352550 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:45.689908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad51d72-bc76-4c08-95d8-fb0cb9352550 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:45.691962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aad51d72-bc76-4c08-95d8-fb0cb9352550 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:45.779534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 493/989] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:45 2022] POST /identity/v3/services => generated 242 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:47.366298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec792d4-6cfb-4bd3-82ad-c8a5bc122e51 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.366854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec792d4-6cfb-4bd3-82ad-c8a5bc122e51 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.366854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ec792d4-6cfb-4bd3-82ad-c8a5bc122e51 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.368295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 497/990] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:47.373384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.373777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.373974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.391022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:47.392207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:47.439345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:47.466925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 494/991] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:47.489984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec792d4-6cfb-4bd3-82ad-c8a5bc122e51 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:47.501395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:47.591832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ec792d4-6cfb-4bd3-82ad-c8a5bc122e51 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:47.626042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:47.626762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.627044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.627324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] PATH_INFO: `/v3/services/object-store` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.628023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:47.629376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-af078433-13e6-481a-9153-e51e5514da6c None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:47.641262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:47.642238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jun 16 11:38:47.643079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:47.647460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 498/992] 10.222.0.40 () {56 vars in 1126 bytes} [Thu Jun 16 11:38:47 2022] GET /identity/v3/services/object-store => generated 93 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:47.648314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:47.741907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91f74c-44a4-4f57-b13b-1f9fb82d8ba2 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.742202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91f74c-44a4-4f57-b13b-1f9fb82d8ba2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.742485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a91f74c-44a4-4f57-b13b-1f9fb82d8ba2 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.747680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 499/993] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:47.748766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.749053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.749332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.766447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01fa6e70-4dd5-4af8-a7b2-c7103e2f8d93 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:47.767228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:47.768485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:47.802438 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:47.803099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.803382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.803662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.804392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:47.804702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:47.806133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:47.811995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 495/994] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:47 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:47.816898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:47.840360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 500/995] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:47 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:47.863344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:47.865573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:47.869388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:47.929058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d0069d9-9746-498f-8d32-1f43cf9afd51 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:47.959316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6159ac69-df97-4a1b-ac71-1281a5fc21f3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:47.963151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:47.963814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.964128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.964404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.965102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:47.965408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:47.966759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2e9f0b-9670-482a-be42-4e280f920439 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:47.972264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 496/996] 10.222.0.40 () {56 vars in 1122 bytes} [Thu Jun 16 11:38:47 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:47.975815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9093c-b2a9-4f36-8624-3c6d2ae7efe1 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.975815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9093c-b2a9-4f36-8624-3c6d2ae7efe1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.975815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bae9093c-b2a9-4f36-8624-3c6d2ae7efe1 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.975815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 497/997] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:47.977806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9093c-b2a9-4f36-8624-3c6d2ae7efe1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:47.996045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:47.996831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:47.997113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:47.997384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] PATH_INFO: `/v3/services/image` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:47.998032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:47.999376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-de0a20d2-f727-4320-8ee2-ef83ef02e08d None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:48.010578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:48.011615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:48.012599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 501/998] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jun 16 11:38:47 2022] GET /identity/v3/services/image => generated 86 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:48.013394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.013394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.013515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.029577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:48.030739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:48.080218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bae9093c-b2a9-4f36-8624-3c6d2ae7efe1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.083214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:48.089693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:48.097491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 502/999] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:48 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:48.098293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:48.115421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.116126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.116650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.117392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.122888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.122888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.218127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 498/1000] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:47 2022] POST /identity/v3/endpoints => generated 325 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:48.218916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:48.271684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b667a1-bf8f-4e39-9935-be19899bfa02 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.321750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.322440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.322726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.323006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.323729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] RBAC: Adding query filter params (name=image) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:48.324090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.325488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.332235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 503/1001] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/services?name=image => generated 122 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:48.338739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:48.344363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48afb935-6a25-4ae9-8331-d6e2e34cabc2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.382797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.383723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.384045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.384320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.384981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.386344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:48.399233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:48.400297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:48.401108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 499/1002] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/services/compute => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:48.404446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:48.406941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:48.439202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:48.449412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50fe1e6e-d768-4f5a-a026-4cf5d9cbd682 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.485807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.486953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.487591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.488111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.489041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] RBAC: Adding query filter params (type=image) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:48.489452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.491088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.503899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 504/1003] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/services?type=image => generated 122 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:48.526291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92eb6940-c311-43d3-bacb-a989977d6520 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.563615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.564357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.564646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.564927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.566196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] RBAC: Adding query filter params (name=compute) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:48.566509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.567943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.578109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 500/1004] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:48.579533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:48.694128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d8d77aa-3c74-4efb-9b08-b67e297fb71d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.715959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:48.730410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.731416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.731843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.732300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.733159 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:48.733633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.735137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.741111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 505/1005] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:48.746477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:48.843857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:48.853196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06826389-05bc-4ee7-b6d3-30211284153c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:48.887650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:48.888348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:48.888634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:48.888913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] PATH_INFO: `/v3/endpoints` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:48.889701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] RBAC: Adding query filter params (interface=public, service_id=ce76d515abae4b5f8aa74ed840ed2bbe, region_id=RegionOne) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:48.890069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:48.891443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f01460d6-3579-4ce8-9c08-82cd4de1664d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:48.901114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 501/1006] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/endpoints?service_id=ce76d515abae4b5f8aa74ed840ed2bbe&interface=public®ion_id=RegionOne => generated 194 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:48.902055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:49.005344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d07c8da-6fbc-478b-bb38-ec0f55bd2796 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:49.039837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-280e9c18-f381-4eb3-9dd0-4b828aa7e616 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:49.040583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9c18-f381-4eb3-9dd0-4b828aa7e616 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:49.040875 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9c18-f381-4eb3-9dd0-4b828aa7e616 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:49.041154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-280e9c18-f381-4eb3-9dd0-4b828aa7e616 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:49.041836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e9c18-f381-4eb3-9dd0-4b828aa7e616 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:49.043207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-280e9c18-f381-4eb3-9dd0-4b828aa7e616 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:49.055916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 506/1007] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:48 2022] GET /identity/v3/services => generated 1727 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:38:50.386868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d507-23cb-4825-baed-f05209c5ba85 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.387396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d507-23cb-4825-baed-f05209c5ba85 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.387511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d841d507-23cb-4825-baed-f05209c5ba85 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.389000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 502/1008] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:50.393971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87a975-6e0c-4320-b952-186371494175 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.394362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87a975-6e0c-4320-b952-186371494175 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.394529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb87a975-6e0c-4320-b952-186371494175 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.411035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-cb87a975-6e0c-4320-b952-186371494175 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:50.412246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cb87a975-6e0c-4320-b952-186371494175 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:50.457692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:50.484001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 507/1009] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:50.504885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d507-23cb-4825-baed-f05209c5ba85 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:50.594584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1c95a-0b20-49f8-84f4-618a8f49653e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.594963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1c95a-0b20-49f8-84f4-618a8f49653e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.595145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a1c95a-0b20-49f8-84f4-618a8f49653e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.599897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 508/1010] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:50 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:50.601928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.602223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.602507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.617541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d841d507-23cb-4825-baed-f05209c5ba85 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:50.620513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:50.621035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:50.653696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:50.654816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.655095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.655366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.656577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:50.657995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:50.660343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 503/1011] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:50.665757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:50.696564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:50.703779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 509/1012] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:50 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:50.727140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:50.773878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0952d932-417c-4cbb-bcdb-e4c85ec8aad4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:50.809857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4507f8-cdc1-43d9-b37b-e7076e0bbb04 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:50.810536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4507f8-cdc1-43d9-b37b-e7076e0bbb04 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.810824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4507f8-cdc1-43d9-b37b-e7076e0bbb04 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.811140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4507f8-cdc1-43d9-b37b-e7076e0bbb04 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.811830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4507f8-cdc1-43d9-b37b-e7076e0bbb04 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:50.813297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4507f8-cdc1-43d9-b37b-e7076e0bbb04 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:50.831237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c74e23e-f5d8-4372-b009-2a5709b268c3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:50.841878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:50.867024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef2950d-8323-4599-8a20-8814d1db2a79 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:50.867695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2950d-8323-4599-8a20-8814d1db2a79 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:50.868040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2950d-8323-4599-8a20-8814d1db2a79 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:50.868595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef2950d-8323-4599-8a20-8814d1db2a79 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:50.869342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2950d-8323-4599-8a20-8814d1db2a79 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:50.870707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef2950d-8323-4599-8a20-8814d1db2a79 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:50.876931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 504/1013] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:50 2022] POST /identity/v3/projects => generated 311 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:50.947994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 510/1014] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:38:50 2022] POST /identity/v3/services => generated 242 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:51.163687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe104b9-4fff-4394-af1a-caa9b68e39cc None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:51.164015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe104b9-4fff-4394-af1a-caa9b68e39cc None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:51.164302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebe104b9-4fff-4394-af1a-caa9b68e39cc None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:51.165863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 505/1015] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:51.171139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:51.171510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:51.171861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:51.189234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:51.190466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:51.265946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 511/1016] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:51 2022] POST /identity/v3/auth/tokens => generated 1803 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:51.289351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe104b9-4fff-4394-af1a-caa9b68e39cc None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:51.370077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:51.394987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebe104b9-4fff-4394-af1a-caa9b68e39cc None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:51.429375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:51.430177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:51.430462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:51.430812 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] PATH_INFO: `/v3/services/compute` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:51.431480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:51.432928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:51.446112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:51.447115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jun 16 11:38:51.448456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:51.452233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 506/1017] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:51 2022] GET /identity/v3/services/compute => generated 88 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:51.453237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:51.545938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:51.553034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f47c611-baa0-4446-b6bb-c997ffa39bec None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:51.587929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:51.588606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:51.588894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:51.589173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:51.589885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] RBAC: Adding query filter params (name=compute) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:51.590192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:51.591547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:51.597489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 512/1018] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:51 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:51.602842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:51.709107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fea243a7-f224-4888-b62c-3e5b5cd4ce36 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:51.742940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:51.743606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:51.743911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:51.744194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:51.744900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] RBAC: Adding query filter params (type=compute) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:51.745208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:51.746561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-156194f5-f343-49eb-abd1-52c1c4f54835 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:51.756856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 507/1019] 10.222.0.40 () {56 vars in 1112 bytes} [Thu Jun 16 11:38:51 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:51.758376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:51.859375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ac309f7-7d5c-48c0-adf0-948075d35049 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:51.893178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4abc57df-d407-4296-a351-8672add3ca60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:51.893833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc57df-d407-4296-a351-8672add3ca60 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:51.894123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc57df-d407-4296-a351-8672add3ca60 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:51.894398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4abc57df-d407-4296-a351-8672add3ca60 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:51.895125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc57df-d407-4296-a351-8672add3ca60 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:51.896526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4abc57df-d407-4296-a351-8672add3ca60 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:51.967266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 513/1020] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:51 2022] POST /identity/v3/endpoints => generated 334 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:38:53.056216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bb38a-a374-44f7-b42d-2818081e2983 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.056734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bb38a-a374-44f7-b42d-2818081e2983 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.056812 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e1bb38a-a374-44f7-b42d-2818081e2983 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.058297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 508/1021] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:53.063337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.063752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.064002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.081717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:53.083355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:53.083800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4df24-ddd8-469b-820d-13a984c07f1b None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.084137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4df24-ddd8-469b-820d-13a984c07f1b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.084419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb4df24-ddd8-469b-820d-13a984c07f1b None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.085797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 509/1022] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:53.090948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.091232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.091511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.097291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.098059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.108234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:53.109509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:53.164763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 514/1023] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:53.182349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 510/1024] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:53 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:53.190681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.205681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.299178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46e5ad79-ad38-4183-9f28-226cf26a60e7 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.314230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-768917d6-0314-431f-b4b5-acda900cee90 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.334065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.334865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.335161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.335442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.336663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.338366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.340181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 515/1025] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:53 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:53.345547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.350455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.351478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.351974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.352422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] PATH_INFO: `/v3/services/image` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.353247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.354898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.355702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.358379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:53.370064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:53.371225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:53.372107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 511/1026] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jun 16 11:38:53 2022] GET /identity/v3/services/image => generated 86 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:53.377280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.449446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-360e4e6c-e5cc-43b4-93e1-24bc0f5c1aa0 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.480753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9cd76f7-2e42-41ff-8950-fe809268c220 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.484910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.485585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.485871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.486149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.487465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.488922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.515082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.515746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.516086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.516371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.517079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] RBAC: Adding query filter params (name=image) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:53.517394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.518784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.523729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 512/1027] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:53 2022] GET /identity/v3/services?name=image => generated 122 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:53.529330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.591983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 516/1028] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:38:53 2022] POST /identity/v3/users => generated 285 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:53.594956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.634079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60389506-eac2-4f93-9f8c-3db0979de487 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.671711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.672451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.672734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.673008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.673730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] RBAC: Adding query filter params (type=image) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:53.674036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.676029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.687083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 513/1029] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:53 2022] GET /identity/v3/services?type=image => generated 350 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:53.689589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.758024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.797926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa15835-fbe3-496b-9060-97afd601345b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.832920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.833597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.833886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.834166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.834965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] RBAC: Adding query filter params (interface=public, service_id=8cd375c2df134f1a960d42fb77d9a2a0, region_id=RegionOne) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:53.835285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.836700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f385d98-b970-4cdd-8cde-300bb59bd717 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.842373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 517/1030] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:38:53 2022] GET /identity/v3/endpoints?service_id=8cd375c2df134f1a960d42fb77d9a2a0&interface=public®ion_id=RegionOne => generated 194 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:53.848798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:53.953302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a99913b6-3b4a-40bc-9cbb-130452e73fb5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:53.979352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:53.988192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54586f44-f9a6-4e5d-8aa8-8bfc4bb0f7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:53.988877 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586f44-f9a6-4e5d-8aa8-8bfc4bb0f7e0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:53.989165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586f44-f9a6-4e5d-8aa8-8bfc4bb0f7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:53.989446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54586f44-f9a6-4e5d-8aa8-8bfc4bb0f7e0 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:53.990133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54586f44-f9a6-4e5d-8aa8-8bfc4bb0f7e0 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:53.991508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54586f44-f9a6-4e5d-8aa8-8bfc4bb0f7e0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:53.998028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 514/1031] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:38:53 2022] GET /identity/v3/services => generated 1957 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:38:54.206417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6776c9-b289-4834-b6e0-4f8272aabc00 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:54.206979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6776c9-b289-4834-b6e0-4f8272aabc00 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:54.207058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a6776c9-b289-4834-b6e0-4f8272aabc00 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:54.208555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 518/1032] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:54.213884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:54.214304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:54.214499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:54.231615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:54.232837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:54.299097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 515/1033] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:54 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:54.322963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6776c9-b289-4834-b6e0-4f8272aabc00 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:54.425411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a6776c9-b289-4834-b6e0-4f8272aabc00 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:54.452112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:54.461066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:54.461827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:54.462370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:54.462652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:54.474101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:54.482598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:54.483589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:54.484497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 519/1034] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 11:38:54 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:54.491030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:54.591766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5225fe0-1082-409f-aa68-fdaaf911ada1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:54.611103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:54.625511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:54.626174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:54.627284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:54.627284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:54.628187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:54.628482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:54.629835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:54.642283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 516/1035] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:38:54 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:54.643295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:54.753278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48be13-4726-4d21-95cd-2afb1899968d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:54.788799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:54.789595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:54.789876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:54.790152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:54.801453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:54.802883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:54.820575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:54.821450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:54.822339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:54.822339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:54.822339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:54.822339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:54.822339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 520/1036] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:54.828654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:54.930136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac298afe-b4cb-4f71-80fd-764d8d65cf01 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:54.963260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:54.963938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:54.964224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:54.964498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:54.965208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:54.965511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:54.966862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:54.979052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 517/1037] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:54.984503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.026177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:55.084337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1788f28b-945b-49f7-8f84-f1bc7e765733 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:55.118561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:55.119356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.119640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.119952 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.127503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:55.128958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:55.142699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:55.143688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:55.145206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 521/1038] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:38:54 2022] GET /identity/v3/users/nova => generated 82 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:55.150105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.194644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:55.251330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e8614ea-2032-4803-a0ad-706a8797b44a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:55.284929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:55.285534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.285816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.286091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.286918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:55.287228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:55.288674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:55.306134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 518/1039] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:38:55 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:55.311523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.407428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:55.411664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ede3a2-e827-40c8-9141-2bdb34bd22dc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:55.444939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ae8dda1-4d33-49f3-a9bc-de89bc3a2928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:55.445597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8dda1-4d33-49f3-a9bc-de89bc3a2928 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.445878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8dda1-4d33-49f3-a9bc-de89bc3a2928 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.446153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ae8dda1-4d33-49f3-a9bc-de89bc3a2928 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.512119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8dda1-4d33-49f3-a9bc-de89bc3a2928 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:55.513570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ae8dda1-4d33-49f3-a9bc-de89bc3a2928 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:55.607129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:55.616230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 522/1040] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:55 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:38:55.620016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.725792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:55.729324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac520ef4-77b3-4c04-9965-c5d1a76e6b52 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:55.767675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:55.769073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.769358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.769637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.780177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:55.781628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:55.799013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:55.799947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:55.800844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:55.800844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:55.800844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:55.800844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:55.801476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b5c9-1ec6-4ee3-9a1e-7c85712f1e11 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.801809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b5c9-1ec6-4ee3-9a1e-7c85712f1e11 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.802054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c6b5c9-1ec6-4ee3-9a1e-7c85712f1e11 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.803937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 519/1042] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:55.804973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 523/1042] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:55.805705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.808604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.808894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.809180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.826504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:55.827702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:55.895685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 524/1043] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:55 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:55.914714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff507483-ff02-4b33-8250-118250fdbbe2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:55.918175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.951149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:55.953047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:55.953047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:55.953047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:55.954056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:55.954374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:55.956386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:55.972525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 520/1044] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:55.973576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:55.988570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.026401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff5809b1-8b51-4265-bc91-9dc585d99cf2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.056962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.063169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.064036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.064322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.064598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.077804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:56.084884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196e227e-d50b-4554-b2a9-8999cf83c6ea None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:56.086681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:56.088087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:56.089191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:56.095485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 525/1045] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:38:56.096531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.127834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.128829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.129173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.129446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.136548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.138008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-371ba4a7-a7ad-4112-a658-3a89b1687c7c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:38:56.153342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:56.154251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:56.155143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 521/1046] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:38:55 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:56.155835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaaea8-234d-4290-bab3-18366baf95ed None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.156097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaaea8-234d-4290-bab3-18366baf95ed None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.156407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9aeaaea8-234d-4290-bab3-18366baf95ed None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.157604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 522/1047] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:56.160131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaaea8-234d-4290-bab3-18366baf95ed None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.205873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f71100b-7cab-40e2-a638-b9c5d54223fa None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.229896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.241072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.241746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.242036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.242318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.247582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:56.247582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.247582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8829cc03-d327-4bc7-9c95-a6d2a7c925b4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.254488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 526/1048] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:56.255830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.256160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.256441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.259987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9aeaaea8-234d-4290-bab3-18366baf95ed None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.271790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:56.273174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:56.293622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.294287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.294576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.294854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.295637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:56.295976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.297398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.308997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 523/1049] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:56.309773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.321164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.340983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 527/1050] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:56 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:56.341782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.410722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366392ce-0bf9-4545-88ff-04552b655c30 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.440896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.445764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.446637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.446924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.447203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] PATH_INFO: `/v3/users/97157083c31040c5b8e38c51750bebf6` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.449490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80d1f1b4-2573-4497-a980-d099002e73c9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.460796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] RBAC: Authorizing `identity:get_user(user_id=97157083c31040c5b8e38c51750bebf6)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.462592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.464463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 524/1051] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/users/97157083c31040c5b8e38c51750bebf6 => generated 285 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:56.465048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.488861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.489604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.489892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.490178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.491021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] RBAC: Adding query filter params (role.id=405b058de2424949a5439d54038e3e48, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=54986964e91b4af7997e09d4682cc31e) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:56.491323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.492775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.498138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 528/1052] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/role_assignments?user.id=54986964e91b4af7997e09d4682cc31e&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=405b058de2424949a5439d54038e3e48 => generated 259 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:56.498908 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.551295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.575581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72191e2a-746a-4bac-bd47-6d22fc12c358 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.614168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79289c23-dc11-4f45-9e21-ffa90ddb91d6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.614535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.615283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.615569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.615861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] PATH_INFO: `/v3/services/image` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.616578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.617982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:56.630417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:56.631470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:56.632353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 525/1053] 10.222.0.40 () {56 vars in 1105 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/services/image => generated 86 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:56.637745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.655075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.655748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.656070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.656354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.705114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.743146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.744800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.749690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b02dd5-3209-47a2-b28f-b519e08dc64f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.790728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.791458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.791739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.792042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.792747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] RBAC: Adding query filter params (name=image) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:56.793065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.794425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.796680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.800206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 526/1054] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/services?name=image => generated 122 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:56.805587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.859163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 529/1055] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 356 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:38:56.860215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.912484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1761efa-0d7d-40f2-912c-318ee69067ab None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:56.929396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:56.946723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:56.947959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:56.947959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:56.947959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:56.948656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] RBAC: Adding query filter params (type=image) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:56.948967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:56.950340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:56.956530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 527/1056] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/services?type=image => generated 350 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:38:56.962955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:56.967922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662aef17-40df-43da-8391-3e81ec9934b2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:57.004677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:57.005650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:57.005942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:57.006221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:57.007402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] RBAC: Authorizing `identity:get_project(project_id=5c83d7395820442c9b2f834c47d395c5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:57.009137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:57.015917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 530/1057] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:38:56 2022] GET /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:57.016754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:57.033437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:57.066657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ad412-7219-4356-a047-44409f2c492f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:57.105849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052211e-b80d-4c68-bb5f-9f960296ddbc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:57.106519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052211e-b80d-4c68-bb5f-9f960296ddbc None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:57.106816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052211e-b80d-4c68-bb5f-9f960296ddbc None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:57.107104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052211e-b80d-4c68-bb5f-9f960296ddbc None admin] PATH_INFO: `/v3/endpoints` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:57.107849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052211e-b80d-4c68-bb5f-9f960296ddbc None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:57.109248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052211e-b80d-4c68-bb5f-9f960296ddbc None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:57.126313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f12ddee-127f-4dd2-b456-ffc2f7ff05c4 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:57.160357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:57.161014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:57.161299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:57.161574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:57.162397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=5c83d7395820442c9b2f834c47d395c5, user.id=97157083c31040c5b8e38c51750bebf6) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:57.162713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:57.164164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b512bafd-8b96-4def-b995-1148139cd1d5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:57.169693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 531/1058] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:38:57 2022] GET /identity/v3/role_assignments?user.id=97157083c31040c5b8e38c51750bebf6&scope.project.id=5c83d7395820442c9b2f834c47d395c5&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:57.173228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 528/1059] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:38:56 2022] POST /identity/v3/endpoints => generated 327 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:38:58.619639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c736-70e1-4001-bd93-7a8caa053ef1 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:58.620233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c736-70e1-4001-bd93-7a8caa053ef1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:58.620233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d28c736-70e1-4001-bd93-7a8caa053ef1 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:58.621636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 532/1060] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:58.626614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:58.626995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:58.627806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:58.644410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:58.645590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:58.680402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:58.727403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 529/1061] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:38:58.750041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28c736-70e1-4001-bd93-7a8caa053ef1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:58.828370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:58.851541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d28c736-70e1-4001-bd93-7a8caa053ef1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:58.886337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:58.887078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:58.887360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:58.887634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:58.897930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:58.906092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:58.907022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:38:58.907896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:58.912047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 533/1062] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 11:38:58 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:58.914221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.013009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66ceba4f-7b16-4d5a-8935-0f942e4655cc None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.047545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.048243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.048530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.048808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.050105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:59.050413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.051783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.059835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 530/1063] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:38:58 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:59.065253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.170288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8392a8fb-72d2-446f-b285-1b6557b83110 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.203706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.204547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.204832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.205108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.213930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.219194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.221555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-65bb0fe2-48f3-40e0-81eb-b53236625b49 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:38:59.240973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:59.241954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:59.242839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:38:59.242839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:38:59.242839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:38:59.242839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:59.243776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 534/1064] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:59.249042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.280911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.284326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084d51f-1802-415a-88d9-95dc377d934e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.284763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084d51f-1802-415a-88d9-95dc377d934e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.285051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3084d51f-1802-415a-88d9-95dc377d934e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.287458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 535/1065] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:59 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:38:59.292137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a995dc-a362-41a9-a758-7528702aadb0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.296156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a995dc-a362-41a9-a758-7528702aadb0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.296156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a995dc-a362-41a9-a758-7528702aadb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.312697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-70a995dc-a362-41a9-a758-7528702aadb0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:59.313984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-70a995dc-a362-41a9-a758-7528702aadb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:59.358199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c98efc16-11b1-49ff-b0f5-5dccd050aff4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.381393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 536/1066] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:59.382746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0d631-2531-421d-bcb1-60c095869625 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.383038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0d631-2531-421d-bcb1-60c095869625 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.383320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0d631-2531-421d-bcb1-60c095869625 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.384702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 537/1067] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:38:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:38:59.391623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.392391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.393209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.393497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.394020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.394223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:59.394377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.394534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.394823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.395923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.407691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 531/1068] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:59.409247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.415325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:38:59.416558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:38:59.483010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 538/1069] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:38:59 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:38:59.483679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.492375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.507297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5bd1367-2870-4f43-a523-9a585c7eb2a6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.543662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.544481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.544771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.545381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.547662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.556228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:38:59.563829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:59.565554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 532/1070] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/roles/admin => generated 83 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:59.566430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.598943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213debd6-3f9c-47a8-aac0-d0146da88918 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.633500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.634302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.634588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.634868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.641803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.643852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:38:59.657935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:38:59.659052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:38:59.660188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:38:59.663853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 539/1071] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/users/nova => generated 82 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:38:59.664689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.665400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b1b5e8d-c02a-4ef6-82ab-fab380c15992 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.702014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.702669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.702955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.703234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.719012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.762874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec332481-24dd-48a7-81d7-463d117c50e5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.770888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.772376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.773721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.796051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.796714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.796999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.797279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.798019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:59.798328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.799754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.808426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 540/1072] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:59.809231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.873669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 533/1073] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:38:59.873669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.906942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d8e61bb-1dac-4646-9d81-6dd49d2bf809 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.914083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:38:59.941784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:38:59.942456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:38:59.942793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:38:59.943082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:38:59.943943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:38:59.944265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:38:59.945688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:38:59.962961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 541/1074] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:38:59.963744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:38:59.971370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc2db7b6-4969-4a8e-9fff-65b8c9b51ede None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:38:59.993459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.007835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.008724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.009010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.009294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] PATH_INFO: `/v3/users/97157083c31040c5b8e38c51750bebf6` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.010479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] RBAC: Authorizing `identity:get_user(user_id=97157083c31040c5b8e38c51750bebf6)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.011938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.013620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 534/1075] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/users/97157083c31040c5b8e38c51750bebf6 => generated 285 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:00.014333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.060731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f9aa1d-5861-4d7b-a81e-524678eb10bc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.095053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.095843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.096157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.096437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.106147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.107686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.111923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42bc1747-a9ef-42d3-9bc6-82b5fef9b15a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.125372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:39:00.125498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:00.126308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:00.127708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:39:00.127708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:39:00.127708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:00.127708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:00.127708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 542/1076] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:38:59 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:00.128036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.147165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.148155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.148437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.148712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.158337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.159824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:39:00.177555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:00.178663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:00.179744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:39:00.179744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:39:00.179744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:00.179744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:00.179744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 535/1077] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:39:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:00.180123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.215613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.233411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd375d51-25f7-4d08-9f3e-b271de2746e5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.267680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.268480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.268767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.269046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.270239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] RBAC: Authorizing `identity:get_project(project_id=5c83d7395820442c9b2f834c47d395c5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.271665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.273398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 543/1078] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:00 2022] GET /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5 => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:00.274151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.278555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f170a1c-34d1-458e-bc6c-53cdebe65867 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.312985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.313640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.313942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.314199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.314947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:00.315254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.316632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.327403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 536/1079] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:00.328176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.362932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.389862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77632c6c-fea7-46c8-9e62-6e5a780bd197 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.429801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.430486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.431648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.431961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.432709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:00.433018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.434418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.446837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 544/1080] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:39:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:00.450946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b62ffbc9-081a-4c0d-976f-f7e3effd132e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.452589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.468007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.493254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.494158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.494436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.494708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5/users/97157083c31040c5b8e38c51750bebf6/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.497430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c83d7395820442c9b2f834c47d395c5, user_id=97157083c31040c5b8e38c51750bebf6, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.499035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.561588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bc74cd-c2b5-4f20-bac1-0c329a07f58c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.577997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 537/1081] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:39:00 2022] PUT /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5/users/97157083c31040c5b8e38c51750bebf6/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:39:00.596581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.599935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.599935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.599935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.606207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.607698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:39:00.624652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:00.625747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:00.626659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 545/1082] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:39:00 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:39:00.632937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.729104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.745095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-626725b1-9f75-474a-89b2-71b5d1cdd366 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.779167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.779835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.780153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.780439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.781260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:00.781574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.783011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b879a738-969d-4fc9-8cf5-3fcf50648b69 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:00.795850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 538/1083] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:39:00 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:00.801723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:00.868540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:00.897922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-422f3ff5-3001-424d-9392-8b6a6840a624 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:00.931586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46308083-ccdb-4b22-a780-0ef8dcd36def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:00.932510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46308083-ccdb-4b22-a780-0ef8dcd36def None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:00.932794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46308083-ccdb-4b22-a780-0ef8dcd36def None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:00.933068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46308083-ccdb-4b22-a780-0ef8dcd36def None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/54986964e91b4af7997e09d4682cc31e/roles/405b058de2424949a5439d54038e3e48` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:00.935215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46308083-ccdb-4b22-a780-0ef8dcd36def None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=54986964e91b4af7997e09d4682cc31e, role_id=405b058de2424949a5439d54038e3e48)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:00.936656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46308083-ccdb-4b22-a780-0ef8dcd36def None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:01.004473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 546/1084] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:39:00 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/54986964e91b4af7997e09d4682cc31e/roles/405b058de2424949a5439d54038e3e48 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:39:02.681043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00c5d4-3771-418d-b2c5-86cb0c4cf2e7 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:02.681572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00c5d4-3771-418d-b2c5-86cb0c4cf2e7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:02.681572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c00c5d4-3771-418d-b2c5-86cb0c4cf2e7 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:02.683009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 539/1085] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:02.688287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:02.688650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:02.688855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:02.705211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:02.706382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:02.771602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 547/1086] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:02 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:02.794529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00c5d4-3771-418d-b2c5-86cb0c4cf2e7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:02.892082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c00c5d4-3771-418d-b2c5-86cb0c4cf2e7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:02.926050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-694ba3b9-ee61-41ca-acb6-5ec5a34182d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:02.926843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ba3b9-ee61-41ca-acb6-5ec5a34182d9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:02.927124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ba3b9-ee61-41ca-acb6-5ec5a34182d9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:02.927397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-694ba3b9-ee61-41ca-acb6-5ec5a34182d9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:02.937219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-694ba3b9-ee61-41ca-acb6-5ec5a34182d9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-694ba3b9-ee61-41ca-acb6-5ec5a34182d9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:02.944746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:02.945852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:02.951528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 540/1087] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:02 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:02.952257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.010330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.052693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c9286ea-e1ac-4e40-93b5-0f444b6cbf0c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.088894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.089568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.089852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.090131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.090877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:03.091189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.092583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.101191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 548/1088] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:02 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:03.103906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf5f10-9fd7-481c-8d50-46cb7a828801 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.104203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf5f10-9fd7-481c-8d50-46cb7a828801 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.104482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ecbf5f10-9fd7-481c-8d50-46cb7a828801 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.105823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 541/1089] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:03.107008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.110620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.110911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.111193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.122415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.127829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:03.129083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:03.195087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 542/1090] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:39:03.206390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d081e0f8-50d6-4940-bc65-5fa1f31cb32b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.219332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.229560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.241250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.242151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.242433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.242705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] PATH_INFO: `/v3/users/97157083c31040c5b8e38c51750bebf6` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.243909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] RBAC: Authorizing `identity:get_user(user_id=97157083c31040c5b8e38c51750bebf6)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.245392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.247009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 549/1091] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/users/97157083c31040c5b8e38c51750bebf6 => generated 285 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:03.253186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.317111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f80b5fef-e633-4e33-bff1-2ba17afe8a55 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.351171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.351829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04132ce1-a548-4777-9840-1aa98a299498 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.352018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.352306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.352583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.355207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.362924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:03.371120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:03.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:03.372951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 543/1092] 10.222.0.40 () {56 vars in 1120 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:39:03.378747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.386905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.387553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.387843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.388156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.444312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.454253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.456356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.476244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e67acaca-66be-4c1b-8708-6cb9e6778831 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.509092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.509786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.510074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.510357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.511102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:03.511416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.512807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.521531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 544/1093] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:03.527251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.554817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 550/1094] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 11:39:03.560452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.584091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.624306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76845a25-3a4a-4ae7-ae1e-d8f0c8e324d7 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.647068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.656240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eb73a12-266a-49a2-8eb8-a806689971d6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.658430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.659223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.659505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.659780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.670047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.671545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:39:03.689039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:03.690939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:03.692409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:39:03.692409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:39:03.692409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:03.692409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:03.692409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 545/1095] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:39:03.696455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.696977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.697835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.698117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.698390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.699590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] RBAC: Authorizing `identity:get_project(project_id=5c83d7395820442c9b2f834c47d395c5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.701045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.707672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 551/1096] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5 => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:03.708849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.792534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8835921f-1a92-4869-8eb2-d3efd6863979 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.802702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:03.804162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb6945d3-bd05-4e02-9f8f-fdb2ab0650e6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.825584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.826235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.826517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.826791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.827502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:03.827808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.829198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.838660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.839299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.839581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.839856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.840700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=5c83d7395820442c9b2f834c47d395c5, user.id=97157083c31040c5b8e38c51750bebf6) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:03.841010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.841681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 546/1097] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:03.842402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:03.847178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:03.848734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 552/1098] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/role_assignments?user.id=97157083c31040c5b8e38c51750bebf6&scope.project.id=5c83d7395820442c9b2f834c47d395c5&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:03.949612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b3921b-d024-4959-a9a2-63b18a3f6887 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:03.986022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:03.987373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:03.987653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:03.987958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] PATH_INFO: `/v3/users/nova` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:03.996135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:03.997604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:39:04.012609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:04.013550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:04.014503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 547/1099] 10.222.0.40 () {56 vars in 1093 bytes} [Thu Jun 16 11:39:03 2022] GET /identity/v3/users/nova => generated 82 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:39:04.020638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:04.046030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:04.123654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0e53c5-4931-4ca8-bd44-a3a9a09b5d31 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:04.159219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:04.159906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:04.160274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:04.160549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:04.161373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:04.161679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:04.163087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:04.180983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 553/1100] 10.222.0.40 () {56 vars in 1133 bytes} [Thu Jun 16 11:39:04 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:04.186741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:04.216519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:04.285686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ded418e-88c4-4efb-8289-5c400a9d7696 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:04.318914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:04.319562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:04.319850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:04.320163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:04.385765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:04.387221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:04.415951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:04.483509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 548/1101] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:04.489254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:04.568004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:04.586643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-056c7bfd-a3e2-4655-abc0-4cb104d3164c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:04.620621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:04.621413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:04.621702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:04.621981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:04.631798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:04.633254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:39:04.650792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:04.651642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:04.652548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:39:04.652548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:39:04.652548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:39:04.652548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:04.656031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 554/1102] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:39:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:04.657612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:04.758862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e471fe1-da46-4c99-b86e-71b7ba276738 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:04.789286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:04.795011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:04.795715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:04.796037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:04.796323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:04.797051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:04.797365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:04.798896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:04.812100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 549/1103] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:39:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:04.834375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:04.919791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-435e90ed-28f0-4a2e-ad3a-eb191a61c26d None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:04.948938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:04.953943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:04.954745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:04.955652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:04.955980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:04.963096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:04.964551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:39:04.978428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:04.979452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:39:04.980443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:04.984032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 555/1104] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:39:04 2022] GET /identity/v3/projects/service => generated 88 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:04.985477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:05.083785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3cbc31-5464-4ad8-ad12-9b17290708c5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:05.116295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:05.116890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:05.117175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:05.117454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:05.118235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:05.118547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:05.119988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08ec12e-bf58-4c42-9806-af54f7539dc3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:05.132074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 550/1105] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:39:04 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:05.138183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:05.235656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5416ef32-3f64-4e25-ba98-96a96f80566a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:05.268688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:05.269334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:05.269627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:05.269907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:05.270736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] RBAC: Adding query filter params (role.id=405b058de2424949a5439d54038e3e48, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=54986964e91b4af7997e09d4682cc31e) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:05.271054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:05.272505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72552d83-47c6-4b16-bb90-59375f7a3e2d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:05.278258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 556/1106] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:05 2022] GET /identity/v3/role_assignments?user.id=54986964e91b4af7997e09d4682cc31e&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=405b058de2424949a5439d54038e3e48 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:05.977156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac02eb-c599-4ba4-a976-29b86ea9d3db None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:05.977700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac02eb-c599-4ba4-a976-29b86ea9d3db None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:05.977700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cac02eb-c599-4ba4-a976-29b86ea9d3db None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:05.979079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 551/1107] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:05.984247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:05.984672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:05.984784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:06.001557 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:06.002824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:06.053934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:06.070411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 557/1108] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:05 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:06.095372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac02eb-c599-4ba4-a976-29b86ea9d3db None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:06.145961 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:06.194668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cac02eb-c599-4ba4-a976-29b86ea9d3db None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:06.228214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1571f4b0-b260-4ce1-bc63-98f4a9d3aa07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:06.228948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f4b0-b260-4ce1-bc63-98f4a9d3aa07 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:06.229232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f4b0-b260-4ce1-bc63-98f4a9d3aa07 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:06.229507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1571f4b0-b260-4ce1-bc63-98f4a9d3aa07 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:06.230671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571f4b0-b260-4ce1-bc63-98f4a9d3aa07 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:06.232121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1571f4b0-b260-4ce1-bc63-98f4a9d3aa07 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:06.233718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 552/1109] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:39:06 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:06.239364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:06.335746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb1dfe03-38ea-4a0e-b8c4-fe4eeb7eb0a3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:06.368261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26486e8e-7434-4680-b513-d5fd697e6361 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:06.368912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486e8e-7434-4680-b513-d5fd697e6361 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:06.369196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486e8e-7434-4680-b513-d5fd697e6361 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:06.369474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26486e8e-7434-4680-b513-d5fd697e6361 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:06.370184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26486e8e-7434-4680-b513-d5fd697e6361 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:06.371599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26486e8e-7434-4680-b513-d5fd697e6361 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:06.478378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 558/1110] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:39:06 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:39:08.600995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3b4b7-749a-40b4-84b4-6d092bb539a9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:08.601547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3b4b7-749a-40b4-84b4-6d092bb539a9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:08.601547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3b4b7-749a-40b4-84b4-6d092bb539a9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:08.602891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 553/1111] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:08.608006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:08.608441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:08.608567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:08.625389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:08.626564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:08.660331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:08.696078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 559/1112] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:08.717254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3b4b7-749a-40b4-84b4-6d092bb539a9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:08.813357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3b4b7-749a-40b4-84b4-6d092bb539a9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:08.829584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:08.846270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:08.847125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:08.847411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:08.847689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] PATH_INFO: `/v3/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:08.849257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] RBAC: Authorizing `identity:get_role(role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:08.850615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:08.852045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 554/1113] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:08 2022] GET /identity/v3/roles/c4622473a5824ed7876be10e3da526ca => generated 223 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:08.857594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:08.951445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4035279c-82b5-4f0b-91a4-cb75a1bd176c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:08.984886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:08.985738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:08.986043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:08.986319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] PATH_INFO: `/v3/users/62b1fdeea13f44b09f0266dfc80b6dcb` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:08.998867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] RBAC: Authorizing `identity:get_user(user_id=62b1fdeea13f44b09f0266dfc80b6dcb)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:09.000317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:09.001959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 560/1114] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:08 2022] GET /identity/v3/users/62b1fdeea13f44b09f0266dfc80b6dcb => generated 286 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:09.007554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:09.103055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a6f758-5eff-4440-add0-db82a2352d9e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:09.134893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:09.135532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:09.135814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:09.136190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:09.188437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:09.200797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:09.202242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:09.296320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 555/1115] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:09.302305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:09.332513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:09.399178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcc185da-0aa0-4a12-9a1c-5e235563adb2 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:09.431020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55fe480b-3239-4f4c-bbd3-abe68db780ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:09.431885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe480b-3239-4f4c-bbd3-abe68db780ca None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:09.432177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe480b-3239-4f4c-bbd3-abe68db780ca None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:09.432453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55fe480b-3239-4f4c-bbd3-abe68db780ca None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:09.433606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe480b-3239-4f4c-bbd3-abe68db780ca None admin] RBAC: Authorizing `identity:get_project(project_id=5c83d7395820442c9b2f834c47d395c5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:09.435048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55fe480b-3239-4f4c-bbd3-abe68db780ca None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:09.436746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 561/1116] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:09 2022] GET /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5 => generated 311 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:09.443439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:09.523925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:09.537907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82c7703c-c6ae-4e6c-a42c-794712895c02 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:09.571357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:09.572034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:09.572315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:09.572589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:09.573403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] RBAC: Adding query filter params (role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=5c83d7395820442c9b2f834c47d395c5, user.id=62b1fdeea13f44b09f0266dfc80b6dcb) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:09.573712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:09.575117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcb994c6-b0ea-4799-8f4e-95781415e2dd None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:09.580924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 556/1117] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:09 2022] GET /identity/v3/role_assignments?user.id=62b1fdeea13f44b09f0266dfc80b6dcb&scope.project.id=5c83d7395820442c9b2f834c47d395c5&role.id=c4622473a5824ed7876be10e3da526ca => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:11.639208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c709ac4-c6d7-4d55-930c-3152b1daed13 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:11.639751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c709ac4-c6d7-4d55-930c-3152b1daed13 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:11.639751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c709ac4-c6d7-4d55-930c-3152b1daed13 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:11.641155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 562/1118] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:11.646786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:11.647199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:11.647337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:11.663004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:11.664208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:11.728061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 557/1119] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:11.750202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c709ac4-c6d7-4d55-930c-3152b1daed13 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:11.844752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:11.849586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c709ac4-c6d7-4d55-930c-3152b1daed13 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:11.882577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:11.883424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:11.883932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:11.884424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] PATH_INFO: `/v3/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:11.886196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] RBAC: Authorizing `identity:get_role(role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:11.887569 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:11.888976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 563/1120] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:11 2022] GET /identity/v3/roles/c4622473a5824ed7876be10e3da526ca => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:11.894873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:11.970798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:11.988939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d02eb43-dd3d-42d8-a518-b43cd73d8270 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:12.021341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:12.022192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:12.022473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:12.022748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] PATH_INFO: `/v3/users/62b1fdeea13f44b09f0266dfc80b6dcb` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:12.023931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] RBAC: Authorizing `identity:get_user(user_id=62b1fdeea13f44b09f0266dfc80b6dcb)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:12.025338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:12.026933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 558/1121] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:11 2022] GET /identity/v3/users/62b1fdeea13f44b09f0266dfc80b6dcb => generated 286 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:12.032526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:12.131265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa3e781-5e7b-41de-b829-f6fafbf2b195 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:12.163760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8167d37-514f-45de-a733-6107f185c694 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:12.164574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8167d37-514f-45de-a733-6107f185c694 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:12.164857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8167d37-514f-45de-a733-6107f185c694 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:12.165129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8167d37-514f-45de-a733-6107f185c694 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:12.166273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8167d37-514f-45de-a733-6107f185c694 None admin] RBAC: Authorizing `identity:get_project(project_id=5c83d7395820442c9b2f834c47d395c5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:12.167681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8167d37-514f-45de-a733-6107f185c694 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:12.169597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 564/1122] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:12 2022] GET /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5 => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:12.175814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:12.271154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53de49fb-2b25-41d7-9557-b0d24d7831b4 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:12.304197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a725dc4e-a58e-463d-9162-b2a00b75ed73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:12.305071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725dc4e-a58e-463d-9162-b2a00b75ed73 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:12.305358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725dc4e-a58e-463d-9162-b2a00b75ed73 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:12.305637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a725dc4e-a58e-463d-9162-b2a00b75ed73 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5/users/62b1fdeea13f44b09f0266dfc80b6dcb/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:12.307818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725dc4e-a58e-463d-9162-b2a00b75ed73 None admin] RBAC: Authorizing `identity:create_grant(project_id=5c83d7395820442c9b2f834c47d395c5, user_id=62b1fdeea13f44b09f0266dfc80b6dcb, role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:12.309263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a725dc4e-a58e-463d-9162-b2a00b75ed73 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:12.380302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 559/1123] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:39:12 2022] PUT /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5/users/62b1fdeea13f44b09f0266dfc80b6dcb/roles/c4622473a5824ed7876be10e3da526ca => generated 0 bytes in 205 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:39:14.439625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9af76-b907-4936-b995-cb3200792c2c None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:14.440186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9af76-b907-4936-b995-cb3200792c2c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:14.440186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e9af76-b907-4936-b995-cb3200792c2c None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:14.441545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 565/1124] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:14.446723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:14.447092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:14.447289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:14.463738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:14.464948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:14.466847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:14.528623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 560/1125] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:14 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:14.552773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9af76-b907-4936-b995-cb3200792c2c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:14.586409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:14.651287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e9af76-b907-4936-b995-cb3200792c2c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:14.685809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:14.686672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:14.686959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:14.687238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] PATH_INFO: `/v3/roles/c4622473a5824ed7876be10e3da526ca` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:14.688725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] RBAC: Authorizing `identity:get_role(role_id=c4622473a5824ed7876be10e3da526ca)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:14.690128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:14.691447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 566/1126] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:14 2022] GET /identity/v3/roles/c4622473a5824ed7876be10e3da526ca => generated 223 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:14.697174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:14.791394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6592bc2-db30-4932-b796-73362892c3e2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:14.823489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:14.824376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:14.824655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:14.824927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] PATH_INFO: `/v3/users/62b1fdeea13f44b09f0266dfc80b6dcb` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:14.826096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] RBAC: Authorizing `identity:get_user(user_id=62b1fdeea13f44b09f0266dfc80b6dcb)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:14.827513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:14.829142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 561/1127] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:14 2022] GET /identity/v3/users/62b1fdeea13f44b09f0266dfc80b6dcb => generated 286 bytes in 132 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:14.834656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:14.931117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09c9b81e-10d3-4b95-919f-b86a92c38402 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:14.932624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:14.964746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:14.965373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:14.965663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:14.965946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:15.030570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:15.032054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:15.125490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 567/1128] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:14 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 291 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:15.131945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:15.140427 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:15.226658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51b200d-d8d2-49fe-9382-2a6109db5723 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:15.259372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb85695-87a4-4e4d-a056-127e505adaf8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:15.260234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb85695-87a4-4e4d-a056-127e505adaf8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:15.260516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb85695-87a4-4e4d-a056-127e505adaf8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:15.260790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb85695-87a4-4e4d-a056-127e505adaf8 None admin] PATH_INFO: `/v3/projects/5c83d7395820442c9b2f834c47d395c5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:15.261942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb85695-87a4-4e4d-a056-127e505adaf8 None admin] RBAC: Authorizing `identity:get_project(project_id=5c83d7395820442c9b2f834c47d395c5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:15.263352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb85695-87a4-4e4d-a056-127e505adaf8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:15.265041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 562/1129] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:15 2022] GET /identity/v3/projects/5c83d7395820442c9b2f834c47d395c5 => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:15.271369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:15.273535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:15.370690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f342e8d4-d840-42de-b860-4338d6490a6a None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:15.402997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:15.403639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:15.403951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:15.404233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:15.405057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] RBAC: Adding query filter params (role.id=c4622473a5824ed7876be10e3da526ca, scope.project.id=5c83d7395820442c9b2f834c47d395c5, user.id=62b1fdeea13f44b09f0266dfc80b6dcb) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:15.405372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:15.406785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e3e055-97cc-4cbe-a0ca-a37868878a10 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:15.413035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 568/1130] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:15 2022] GET /identity/v3/role_assignments?user.id=62b1fdeea13f44b09f0266dfc80b6dcb&scope.project.id=5c83d7395820442c9b2f834c47d395c5&role.id=c4622473a5824ed7876be10e3da526ca => generated 608 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:17.500511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ecadf-7853-4156-9195-cda83cb66dcb None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:17.501040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ecadf-7853-4156-9195-cda83cb66dcb None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:17.501040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e02ecadf-7853-4156-9195-cda83cb66dcb None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:17.502917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 563/1131] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:17.507989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:17.508382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:17.508548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:17.525303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:17.526465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:17.582934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:17.590209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 569/1132] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:17 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:39:17.612776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ecadf-7853-4156-9195-cda83cb66dcb None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:17.684678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:17.710122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e02ecadf-7853-4156-9195-cda83cb66dcb None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:17.742605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a4856d-afcc-48af-9814-970f09d83ad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:17.743387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4856d-afcc-48af-9814-970f09d83ad8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:17.743671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4856d-afcc-48af-9814-970f09d83ad8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:17.743976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a4856d-afcc-48af-9814-970f09d83ad8 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:17.745143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4856d-afcc-48af-9814-970f09d83ad8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:17.746560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39a4856d-afcc-48af-9814-970f09d83ad8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:17.748200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 564/1133] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:39:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:17.754779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:17.850224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d9177d6-0e02-452a-a06c-85b7289af893 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:17.883258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0fd46d-ae90-459a-8cb8-cdf50c38b83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:17.883931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fd46d-ae90-459a-8cb8-cdf50c38b83a None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:17.884218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fd46d-ae90-459a-8cb8-cdf50c38b83a None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:17.884495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0fd46d-ae90-459a-8cb8-cdf50c38b83a None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:17.885208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fd46d-ae90-459a-8cb8-cdf50c38b83a None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:17.886625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f0fd46d-ae90-459a-8cb8-cdf50c38b83a None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:17.993605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 570/1134] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:39:17 2022] POST /identity/v3/projects => generated 311 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:39:20.117060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d7c4-f293-49bb-84cb-8eb685542689 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:20.117622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d7c4-f293-49bb-84cb-8eb685542689 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:20.117622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0416d7c4-f293-49bb-84cb-8eb685542689 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:20.118952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 565/1135] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:20.124064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:20.130408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:20.130408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:20.141531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:20.142734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:20.185710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:20.208300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 571/1136] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:20.232797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416d7c4-f293-49bb-84cb-8eb685542689 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:20.329660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0416d7c4-f293-49bb-84cb-8eb685542689 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:20.361925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ab7117-9786-44d0-b1e9-187f4e3b5b25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:20.362699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab7117-9786-44d0-b1e9-187f4e3b5b25 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:20.362982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab7117-9786-44d0-b1e9-187f4e3b5b25 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:20.363256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ab7117-9786-44d0-b1e9-187f4e3b5b25 None admin] PATH_INFO: `/v3/domains/default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:20.364434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab7117-9786-44d0-b1e9-187f4e3b5b25 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:20.365850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ab7117-9786-44d0-b1e9-187f4e3b5b25 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:20.367482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 566/1137] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:39:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:20.373115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:20.467568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48d5afee-7fae-498b-8cf5-2578e18cf262 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:20.500545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f3d174a-9eee-49a6-be2b-3b2b63713827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:20.501232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d174a-9eee-49a6-be2b-3b2b63713827 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:20.501518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d174a-9eee-49a6-be2b-3b2b63713827 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:20.501799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f3d174a-9eee-49a6-be2b-3b2b63713827 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:20.502518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d174a-9eee-49a6-be2b-3b2b63713827 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:20.503966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f3d174a-9eee-49a6-be2b-3b2b63713827 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:20.588272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 572/1138] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:39:20 2022] POST /identity/v3/users => generated 286 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:39:22.700221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62af1dd-0644-4929-8a58-d13c1f73bc10 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:22.700636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62af1dd-0644-4929-8a58-d13c1f73bc10 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:22.700773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62af1dd-0644-4929-8a58-d13c1f73bc10 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:22.702127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 567/1139] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:22.707671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:22.708084 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:22.708265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:22.725353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:22.726527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:22.752303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:22.789719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 573/1140] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:22 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:22.812290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62af1dd-0644-4929-8a58-d13c1f73bc10 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:22.825058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:22.912036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62af1dd-0644-4929-8a58-d13c1f73bc10 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:22.944202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:22.944926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:22.945212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:22.945491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:22.955786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:22.963643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:22.964776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:22.965821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 568/1141] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:22.970662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:23.065087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b71d7e-bc92-4b54-aeb4-c5f4417a01c3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:23.097746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:23.098399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:23.098685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:23.098964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:23.099703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:23.100043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:23.101393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:23.109119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 574/1142] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:22 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:23.115085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:23.181522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:23.211069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83e9d2b4-1d5c-4304-8c4a-e308218dccdc None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:23.243251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:23.244136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:23.244422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:23.244699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] PATH_INFO: `/v3/users/5c87dc90d68042a38497788b4d036b82` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:23.257868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] RBAC: Authorizing `identity:get_user(user_id=5c87dc90d68042a38497788b4d036b82)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:23.259297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:23.264168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 569/1143] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:23 2022] GET /identity/v3/users/5c87dc90d68042a38497788b4d036b82 => generated 286 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:23.266575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:23.293893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:23.366165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7ceccdc-fb6e-4be0-ae08-79fb5df65ef0 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:23.399667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:23.400350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:23.400640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:23.400944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:23.468020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:23.469403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:23.495723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:23.566316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 575/1144] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:23.572227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:23.667290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff6554-9661-42c8-94bf-0950c98cba9f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:23.692833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:23.700349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ccaed9d-3383-4a14-adb1-2de8efd4c527 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:23.704314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaed9d-3383-4a14-adb1-2de8efd4c527 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:23.704314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaed9d-3383-4a14-adb1-2de8efd4c527 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:23.704314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ccaed9d-3383-4a14-adb1-2de8efd4c527 None admin] PATH_INFO: `/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:23.704314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaed9d-3383-4a14-adb1-2de8efd4c527 None admin] RBAC: Authorizing `identity:get_project(project_id=40c7631c652c4e3b8b9bc35bbc1e9baf)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:23.704638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ccaed9d-3383-4a14-adb1-2de8efd4c527 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:23.706013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 570/1145] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:23 2022] GET /identity/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:23.712416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:23.807737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b5ed685-4719-4aa4-9066-cbb862fd2613 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:23.831321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:23.839676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:23.840346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:23.840630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:23.840903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:23.841760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=40c7631c652c4e3b8b9bc35bbc1e9baf, user.id=5c87dc90d68042a38497788b4d036b82) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:23.842070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:23.843502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3ed3c6c-720f-4d26-8e0b-573ac6402c5d None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:23.849329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 576/1146] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:23 2022] GET /identity/v3/role_assignments?user.id=5c87dc90d68042a38497788b4d036b82&scope.project.id=40c7631c652c4e3b8b9bc35bbc1e9baf&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:25.890885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f877be2-2bb7-4853-85a9-71a7e0341015 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:25.891392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f877be2-2bb7-4853-85a9-71a7e0341015 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:25.891517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f877be2-2bb7-4853-85a9-71a7e0341015 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:25.892830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 571/1147] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:25.897827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:25.898243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:25.898389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:25.915182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:25.916411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:25.981342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 577/1148] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:26.003424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f877be2-2bb7-4853-85a9-71a7e0341015 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:26.101454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f877be2-2bb7-4853-85a9-71a7e0341015 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:26.133820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:26.134597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:26.134883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:26.135161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:26.145415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:26.153008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:26.154053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:26.155085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 572/1149] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 151 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:26.160859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:26.254348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fd31bc1-517a-41ff-861f-dba5e6e2d321 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:26.273485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:26.287048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:26.287702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:26.288601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:26.288601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:26.289081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:26.289389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:26.290743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:26.298408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 578/1150] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:26.303981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:26.358826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:26.400417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8e41d85-df71-463e-8d2d-6ef72aefe56b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:26.432387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:26.433247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:26.433531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:26.433809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] PATH_INFO: `/v3/users/5c87dc90d68042a38497788b4d036b82` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:26.434983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] RBAC: Authorizing `identity:get_user(user_id=5c87dc90d68042a38497788b4d036b82)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:26.436433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:26.438032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 573/1151] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:26 2022] GET /identity/v3/users/5c87dc90d68042a38497788b4d036b82 => generated 286 bytes in 135 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:26.443712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:26.542555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3992b93-0909-49fa-8e89-d1d867ce1197 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:26.574639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d60fe3-7db2-4805-b552-702f361bc7cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:26.575493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d60fe3-7db2-4805-b552-702f361bc7cc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:26.575778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d60fe3-7db2-4805-b552-702f361bc7cc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:26.576080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d60fe3-7db2-4805-b552-702f361bc7cc None admin] PATH_INFO: `/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:26.577244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d60fe3-7db2-4805-b552-702f361bc7cc None admin] RBAC: Authorizing `identity:get_project(project_id=40c7631c652c4e3b8b9bc35bbc1e9baf)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:26.578668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d60fe3-7db2-4805-b552-702f361bc7cc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:26.580988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 579/1152] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:26 2022] GET /identity/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf => generated 311 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:26.586874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:26.682805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2efa258e-9a63-4c2a-aa46-e9b99e99c244 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:26.704563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:26.714913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b7d4c0a-7012-4efd-93c4-74ae00e2de40 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:26.715789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d4c0a-7012-4efd-93c4-74ae00e2de40 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:26.716100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d4c0a-7012-4efd-93c4-74ae00e2de40 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:26.720077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b7d4c0a-7012-4efd-93c4-74ae00e2de40 None admin] PATH_INFO: `/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf/users/5c87dc90d68042a38497788b4d036b82/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:26.720077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d4c0a-7012-4efd-93c4-74ae00e2de40 None admin] RBAC: Authorizing `identity:create_grant(project_id=40c7631c652c4e3b8b9bc35bbc1e9baf, user_id=5c87dc90d68042a38497788b4d036b82, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:26.720432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b7d4c0a-7012-4efd-93c4-74ae00e2de40 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:26.804769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 574/1153] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:39:26 2022] PUT /identity/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf/users/5c87dc90d68042a38497788b4d036b82/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:39:28.857670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64e787-ac39-44d9-9e68-672d4d4de773 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:28.858183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64e787-ac39-44d9-9e68-672d4d4de773 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:28.858268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f64e787-ac39-44d9-9e68-672d4d4de773 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:28.859594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 580/1154] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:28.864643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:28.865079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:28.865206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:28.882057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:28.883222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:28.946552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 575/1155] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:28.970473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64e787-ac39-44d9-9e68-672d4d4de773 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:29.021143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:29.069918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f64e787-ac39-44d9-9e68-672d4d4de773 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:29.102400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:29.103182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:29.103472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:29.103754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:29.114039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:29.121650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:29.122715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:29.123563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 581/1156] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:29.129300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:29.226158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad89452-af2a-41a0-9bfc-54cd9ce14f72 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:29.240102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:29.259769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:29.260461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:29.260749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:29.261031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:29.261780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:29.262090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:29.263452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:29.271190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 576/1157] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:29.276973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:29.372638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76fd3bc7-5bb3-4eb8-bb2a-36fee03daece None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:29.383038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:29.404917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:29.405868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:29.406150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:29.406426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] PATH_INFO: `/v3/users/5c87dc90d68042a38497788b4d036b82` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:29.407606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] RBAC: Authorizing `identity:get_user(user_id=5c87dc90d68042a38497788b4d036b82)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:29.409057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:29.411148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 582/1158] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:29 2022] GET /identity/v3/users/5c87dc90d68042a38497788b4d036b82 => generated 286 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:29.416486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:29.513242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c84ad5d6-35fd-409e-8bef-769bea41ba8c None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:29.545364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:29.546002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:29.546289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:29.546566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:29.592591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:29.614900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:29.616387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:29.710127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 577/1159] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:29.716330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:29.811365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4134008a-c230-4aee-aa83-1eee381de9a3 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:29.842836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08cbe711-90b4-4865-9118-a488ee8f3711 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:29.843678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbe711-90b4-4865-9118-a488ee8f3711 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:29.843989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbe711-90b4-4865-9118-a488ee8f3711 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:29.844263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08cbe711-90b4-4865-9118-a488ee8f3711 None admin] PATH_INFO: `/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:29.845920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbe711-90b4-4865-9118-a488ee8f3711 None admin] RBAC: Authorizing `identity:get_project(project_id=40c7631c652c4e3b8b9bc35bbc1e9baf)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:29.847329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08cbe711-90b4-4865-9118-a488ee8f3711 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:29.848994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 583/1160] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:29 2022] GET /identity/v3/projects/40c7631c652c4e3b8b9bc35bbc1e9baf => generated 311 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:29.854773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:29.927614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:29.948661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2413959a-b52a-472b-85ff-79177d22231b None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:29.982230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:29.982867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:29.983150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:29.983426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:29.984270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=40c7631c652c4e3b8b9bc35bbc1e9baf, user.id=5c87dc90d68042a38497788b4d036b82) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:29.984584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:29.986023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a7061d-7067-4293-8f97-1135bcc621e0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:29.991603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 578/1161] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:29 2022] GET /identity/v3/role_assignments?user.id=5c87dc90d68042a38497788b4d036b82&scope.project.id=40c7631c652c4e3b8b9bc35bbc1e9baf&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:32.089087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294ad10-e954-4c52-a075-3bd4e6eb152e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:32.089608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294ad10-e954-4c52-a075-3bd4e6eb152e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:32.089699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b294ad10-e954-4c52-a075-3bd4e6eb152e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:32.090992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 584/1162] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:32.096187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:32.096562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:32.096751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:32.113526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:32.114746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:32.179144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 579/1163] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:32.201380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294ad10-e954-4c52-a075-3bd4e6eb152e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:32.248518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:32.299783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b294ad10-e954-4c52-a075-3bd4e6eb152e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:32.332025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:32.332664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:32.332951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:32.333231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:32.397032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:32.398407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:32.441814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:32.493675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 585/1164] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:32 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 293 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:32.499770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:32.565894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:32.595413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b22df4-be99-4041-aa67-c7c864e453b2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:32.627875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:32.628675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:32.628958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:32.629231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:32.635643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:32.637106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-06ec0c8f-9e2e-4aa5-9bc3-c4744ca68852 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:32.651537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:32.652723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:32.653655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 580/1165] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:39:32 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:32.658552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:32.753995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfd2145f-8df0-4029-81dc-28a5eb31c3fc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:32.787158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:32.787815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:32.788128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:32.788406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:32.789121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:32.789426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:32.790811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38aff471-e56b-4cbc-81c0-bcc3085502c0 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:32.797806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 586/1166] 10.222.0.40 () {56 vars in 1115 bytes} [Thu Jun 16 11:39:32 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:34.850454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9f863-9c92-4ba3-840d-243494fda876 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:34.850995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9f863-9c92-4ba3-840d-243494fda876 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:34.850995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07e9f863-9c92-4ba3-840d-243494fda876 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:34.852414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 581/1167] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:34.857371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb22e1-73ba-4ee9-9766-8e1cee9c7992 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:34.857734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb22e1-73ba-4ee9-9766-8e1cee9c7992 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:34.857936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bbb22e1-73ba-4ee9-9766-8e1cee9c7992 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:34.874668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2bbb22e1-73ba-4ee9-9766-8e1cee9c7992 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:34.875856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2bbb22e1-73ba-4ee9-9766-8e1cee9c7992 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:34.878532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:34.938976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 587/1168] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:34.960750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9f863-9c92-4ba3-840d-243494fda876 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:35.060068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07e9f863-9c92-4ba3-840d-243494fda876 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:35.086626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:35.092395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e61fa4-4217-4e08-9ca3-17bc138de818 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:35.093047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e61fa4-4217-4e08-9ca3-17bc138de818 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:35.093360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e61fa4-4217-4e08-9ca3-17bc138de818 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:35.093641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e61fa4-4217-4e08-9ca3-17bc138de818 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:35.094918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e61fa4-4217-4e08-9ca3-17bc138de818 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:35.096310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e61fa4-4217-4e08-9ca3-17bc138de818 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:35.183081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 582/1169] 10.222.0.40 () {60 vars in 1138 bytes} [Thu Jun 16 11:39:34 2022] POST /identity/v3/domains => generated 267 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:39:37.305162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39348a-ba5d-455d-a749-1136f2de45dd None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:37.305737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39348a-ba5d-455d-a749-1136f2de45dd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:37.305737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc39348a-ba5d-455d-a749-1136f2de45dd None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:37.307077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 588/1170] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:37.312154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:37.312550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:37.313243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:37.330051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:37.331218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:37.394915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 583/1171] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:39:37.417860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39348a-ba5d-455d-a749-1136f2de45dd None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:37.515993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc39348a-ba5d-455d-a749-1136f2de45dd None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:37.548272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662f8148-be26-4291-9cd0-8b17c4eac77e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:37.549119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f8148-be26-4291-9cd0-8b17c4eac77e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:37.549403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f8148-be26-4291-9cd0-8b17c4eac77e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:37.549680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662f8148-be26-4291-9cd0-8b17c4eac77e None admin] PATH_INFO: `/v3/domains/3c20cee3b2364a289e5b0170fa8f9072` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:37.557588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:37.559963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f8148-be26-4291-9cd0-8b17c4eac77e None admin] RBAC: Authorizing `identity:get_domain(domain_id=3c20cee3b2364a289e5b0170fa8f9072)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:37.561397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662f8148-be26-4291-9cd0-8b17c4eac77e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:37.563569 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 589/1172] 10.222.0.40 () {56 vars in 1183 bytes} [Thu Jun 16 11:39:37 2022] GET /identity/v3/domains/3c20cee3b2364a289e5b0170fa8f9072 => generated 267 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:37.569308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:37.663945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63847b6e-9fc1-46f4-918b-9c0739e729d8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:37.683504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:37.697570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12f8ce2d-f965-4be0-a11e-c8ee67e576b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:37.698238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8ce2d-f965-4be0-a11e-c8ee67e576b1 None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:37.698523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8ce2d-f965-4be0-a11e-c8ee67e576b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:37.698802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12f8ce2d-f965-4be0-a11e-c8ee67e576b1 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:37.699518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8ce2d-f965-4be0-a11e-c8ee67e576b1 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:37.701089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12f8ce2d-f965-4be0-a11e-c8ee67e576b1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:37.765572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 584/1173] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:39:37 2022] POST /identity/v3/projects => generated 361 bytes in 197 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:39:39.888931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845c55-9dec-4008-a475-62470a24083b None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:39.889463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845c55-9dec-4008-a475-62470a24083b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:39.889562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13845c55-9dec-4008-a475-62470a24083b None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:39.891377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 590/1174] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:39.896480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:39.896859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:39.897038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:39.913898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:39.915065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:39.978722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 585/1175] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:39 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:40.003324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845c55-9dec-4008-a475-62470a24083b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:40.100949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13845c55-9dec-4008-a475-62470a24083b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:40.133314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ccaa4dc-98af-45e7-8288-1c92594a2ae2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:40.134157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa4dc-98af-45e7-8288-1c92594a2ae2 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:40.134443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa4dc-98af-45e7-8288-1c92594a2ae2 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:40.134722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ccaa4dc-98af-45e7-8288-1c92594a2ae2 None admin] PATH_INFO: `/v3/domains/3c20cee3b2364a289e5b0170fa8f9072` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:40.135891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaa4dc-98af-45e7-8288-1c92594a2ae2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=3c20cee3b2364a289e5b0170fa8f9072)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:40.137323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ccaa4dc-98af-45e7-8288-1c92594a2ae2 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:40.138899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 591/1176] 10.222.0.40 () {56 vars in 1183 bytes} [Thu Jun 16 11:39:40 2022] GET /identity/v3/domains/3c20cee3b2364a289e5b0170fa8f9072 => generated 267 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:40.145245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:40.238820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00ce468-e56e-474b-a319-471acfc82d55 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:40.240263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:40.272032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6846e610-15e4-4e82-8c2c-9c687189d78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:40.272746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e610-15e4-4e82-8c2c-9c687189d78c None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:40.273272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e610-15e4-4e82-8c2c-9c687189d78c None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:40.273608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6846e610-15e4-4e82-8c2c-9c687189d78c None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:40.274469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846e610-15e4-4e82-8c2c-9c687189d78c None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:40.276371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6846e610-15e4-4e82-8c2c-9c687189d78c None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:40.387377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 586/1177] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:39:40 2022] POST /identity/v3/users => generated 311 bytes in 243 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:39:42.529887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e3745-f53f-4e3b-ad92-97b5c79a2727 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:42.530416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e3745-f53f-4e3b-ad92-97b5c79a2727 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:42.530530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662e3745-f53f-4e3b-ad92-97b5c79a2727 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:42.531833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 592/1178] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:42.537020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:42.537406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:42.537572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:42.553816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:42.554983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:42.620761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 587/1179] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:42 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:42.644369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e3745-f53f-4e3b-ad92-97b5c79a2727 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:42.710527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:42.743493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662e3745-f53f-4e3b-ad92-97b5c79a2727 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:42.776638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:42.777415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:42.777697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:42.777970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:42.787633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:42.795072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:42.796107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:42.797576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 593/1180] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:42 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:42.802725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:42.886117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:42.897449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f075b8b5-5841-4fe3-93b8-9f281ced77f1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:42.929919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:42.930585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:42.930867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:42.931158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:42.931924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:42.932237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:42.933604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:42.942061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 588/1181] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:42 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:42.947604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:43.043327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8c3397-cb6c-45aa-961c-b7d1a74e0c98 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:43.068059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:43.075449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:43.077946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:43.077946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:43.077946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] PATH_INFO: `/v3/users/717898af2c8b4adb9c9efc9820fdc66a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:43.090050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] RBAC: Authorizing `identity:get_user(user_id=717898af2c8b4adb9c9efc9820fdc66a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:43.091473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:43.093110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 594/1182] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:42 2022] GET /identity/v3/users/717898af2c8b4adb9c9efc9820fdc66a => generated 311 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:43.099388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:43.195712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e271b123-e163-402c-9710-cdc109ebf1c9 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:43.231457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:43.232136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:43.232419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:43.232696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:43.250838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:43.297912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:43.299329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:43.393003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 589/1183] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:43 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jun 16 11:39:43.398824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:43.493671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f6955-325a-4cdd-a181-5f24c52cea6f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:43.525873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7062c514-c8f6-44aa-af10-e163dddfe828 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:43.526710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062c514-c8f6-44aa-af10-e163dddfe828 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:43.527019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062c514-c8f6-44aa-af10-e163dddfe828 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:43.527294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7062c514-c8f6-44aa-af10-e163dddfe828 None admin] PATH_INFO: `/v3/projects/e61a4dfe18704c7da7edc2c3460b2830` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:43.528482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062c514-c8f6-44aa-af10-e163dddfe828 None admin] RBAC: Authorizing `identity:get_project(project_id=e61a4dfe18704c7da7edc2c3460b2830)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:43.529897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7062c514-c8f6-44aa-af10-e163dddfe828 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:43.532168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 595/1184] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:43 2022] GET /identity/v3/projects/e61a4dfe18704c7da7edc2c3460b2830 => generated 361 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:43.537936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:43.584708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:43.631512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15027128-e659-4839-9dc5-b784ab3cc242 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:43.664094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:43.664740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:43.665020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:43.665295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:43.666108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=e61a4dfe18704c7da7edc2c3460b2830, user.id=717898af2c8b4adb9c9efc9820fdc66a) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:43.666419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:43.667827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf71797-ddab-4dd2-87c0-b2f7a48efd89 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:43.673241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 590/1185] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:43 2022] GET /identity/v3/role_assignments?user.id=717898af2c8b4adb9c9efc9820fdc66a&scope.project.id=e61a4dfe18704c7da7edc2c3460b2830&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:45.755102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963548-9746-4c94-a891-767515c48a19 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:45.755648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963548-9746-4c94-a891-767515c48a19 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:45.755648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5963548-9746-4c94-a891-767515c48a19 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:45.757042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 596/1186] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:45.762496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:45.762934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:45.763035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:45.779347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:45.780548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:45.843299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 591/1187] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:45 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:39:45.865589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963548-9746-4c94-a891-767515c48a19 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:45.919734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:45.963844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5963548-9746-4c94-a891-767515c48a19 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:45.997067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:45.997847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:45.998131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:45.998407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:46.008211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:46.015773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:46.016886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:39:46.017895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 597/1188] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:46.022893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:46.066300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:46.117849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44af60ce-6276-4739-8db0-96383e2aa599 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:46.149716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:46.150367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:46.150648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:46.150922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:46.151661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:46.151994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:46.153395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:46.161598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 592/1189] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:46 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:46.167081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:46.262820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e415e909-fea6-485f-8ba1-f3ebf2baad2b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:46.275595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:46.294765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:46.295618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:46.295926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:46.296209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] PATH_INFO: `/v3/users/717898af2c8b4adb9c9efc9820fdc66a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:46.297968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] RBAC: Authorizing `identity:get_user(user_id=717898af2c8b4adb9c9efc9820fdc66a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:46.299384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:46.301000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 598/1190] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:46 2022] GET /identity/v3/users/717898af2c8b4adb9c9efc9820fdc66a => generated 311 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:46.306617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:46.401292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1184d700-7157-4d83-8f6c-f2224a6b330d None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:46.408501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:46.433413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f57b1a5f-b46c-4410-b10f-c4eb769e5c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:46.434261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b1a5f-b46c-4410-b10f-c4eb769e5c89 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:46.434542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b1a5f-b46c-4410-b10f-c4eb769e5c89 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:46.434813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f57b1a5f-b46c-4410-b10f-c4eb769e5c89 None admin] PATH_INFO: `/v3/projects/e61a4dfe18704c7da7edc2c3460b2830` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:46.435997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b1a5f-b46c-4410-b10f-c4eb769e5c89 None admin] RBAC: Authorizing `identity:get_project(project_id=e61a4dfe18704c7da7edc2c3460b2830)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:46.437440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f57b1a5f-b46c-4410-b10f-c4eb769e5c89 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:46.439102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 593/1191] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:46 2022] GET /identity/v3/projects/e61a4dfe18704c7da7edc2c3460b2830 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:46.444977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:46.542364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f91f9c-ddb4-46b5-a223-f5634e91e6bd None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:46.574441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1984305c-9666-4160-a881-da3152cab468 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:46.575316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984305c-9666-4160-a881-da3152cab468 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:46.575607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984305c-9666-4160-a881-da3152cab468 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:46.575905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1984305c-9666-4160-a881-da3152cab468 None admin] PATH_INFO: `/v3/projects/e61a4dfe18704c7da7edc2c3460b2830/users/717898af2c8b4adb9c9efc9820fdc66a/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:46.577844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984305c-9666-4160-a881-da3152cab468 None admin] RBAC: Authorizing `identity:create_grant(project_id=e61a4dfe18704c7da7edc2c3460b2830, user_id=717898af2c8b4adb9c9efc9820fdc66a, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:46.579769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1984305c-9666-4160-a881-da3152cab468 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:46.638727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 599/1192] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:39:46 2022] PUT /identity/v3/projects/e61a4dfe18704c7da7edc2c3460b2830/users/717898af2c8b4adb9c9efc9820fdc66a/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:39:48.693619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab884b7-a4d8-4a9c-bbae-74a2613b5695 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:48.694088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab884b7-a4d8-4a9c-bbae-74a2613b5695 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:48.694185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ab884b7-a4d8-4a9c-bbae-74a2613b5695 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:48.695515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 594/1193] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:39:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:39:48.700566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:48.700996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:48.701137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:48.718193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:39:48.720016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:39:48.753228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:48.783899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 600/1194] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:39:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:39:48.806569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab884b7-a4d8-4a9c-bbae-74a2613b5695 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:48.905179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ab884b7-a4d8-4a9c-bbae-74a2613b5695 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:48.937144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:48.937972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:48.938207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:48.938485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:48.948629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:39:48.956719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:39:48.957778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:39:48.958797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 595/1195] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:39:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 152 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:39:48.963956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:49.058540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d0d3dd-4012-40d8-bfe9-9dae03083c4d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:49.092300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:49.092953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:49.093237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:49.093512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:49.094259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:49.094573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:49.095951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:49.103624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 601/1196] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:39:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:39:49.109032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:49.141866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:49.205935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7f056a-73b8-4823-8be6-7dbcdc151e1f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:49.240150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:49.241002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:49.241286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:49.241564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] PATH_INFO: `/v3/users/717898af2c8b4adb9c9efc9820fdc66a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:49.242733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] RBAC: Authorizing `identity:get_user(user_id=717898af2c8b4adb9c9efc9820fdc66a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:49.244187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:49.245783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 596/1197] 10.222.0.40 () {56 vars in 1177 bytes} [Thu Jun 16 11:39:49 2022] GET /identity/v3/users/717898af2c8b4adb9c9efc9820fdc66a => generated 311 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:49.251368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:49.284671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:49.347163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3aa19bb-9d4a-42b1-8055-9016a37ca22e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:49.379043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:49.379674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:49.379981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:49.380261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:49.443811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:49.445263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:49.477099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:49.539572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 602/1198] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:39:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:39:49.545237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:49.638387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d543a504-60e3-43f5-bc83-ca3d0a737c75 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:49.639862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:49.671556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be9a4fa-55b1-473c-bdbd-8da23d8fece6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:49.672958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9a4fa-55b1-473c-bdbd-8da23d8fece6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:49.673383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9a4fa-55b1-473c-bdbd-8da23d8fece6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:49.673763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be9a4fa-55b1-473c-bdbd-8da23d8fece6 None admin] PATH_INFO: `/v3/projects/e61a4dfe18704c7da7edc2c3460b2830` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:49.675005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9a4fa-55b1-473c-bdbd-8da23d8fece6 None admin] RBAC: Authorizing `identity:get_project(project_id=e61a4dfe18704c7da7edc2c3460b2830)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:49.676464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be9a4fa-55b1-473c-bdbd-8da23d8fece6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:49.678316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 597/1199] 10.222.0.40 () {56 vars in 1186 bytes} [Thu Jun 16 11:39:49 2022] GET /identity/v3/projects/e61a4dfe18704c7da7edc2c3460b2830 => generated 361 bytes in 133 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:49.684309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:39:49.780456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99124e7e-ba97-4dd7-b4cb-c211358b9d27 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:39:49.811892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:39:49.813057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:39:49.813342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:39:49.813620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:39:49.814445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=e61a4dfe18704c7da7edc2c3460b2830, user.id=717898af2c8b4adb9c9efc9820fdc66a) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:39:49.814760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:39:49.816198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96979157-2925-4ea8-bc40-fbfb3f33bbae None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:39:49.817986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:49.822067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 603/1200] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:39:49 2022] GET /identity/v3/role_assignments?user.id=717898af2c8b4adb9c9efc9820fdc66a&scope.project.id=e61a4dfe18704c7da7edc2c3460b2830&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:39:59.667754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=15 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:39:59.856022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:00.698319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6d786-c616-41a4-88bb-ed347d396bc9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:00.698764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6d786-c616-41a4-88bb-ed347d396bc9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:00.698862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4f6d786-c616-41a4-88bb-ed347d396bc9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:00.700494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 598/1201] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:00.705741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:00.706189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:00.706301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:00.723213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:00.724427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:00.789785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 604/1202] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:00 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:00.814589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6d786-c616-41a4-88bb-ed347d396bc9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:00.915887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4f6d786-c616-41a4-88bb-ed347d396bc9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:00.949819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:00.950634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:00.950923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:00.951207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:00.962227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:00.963670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:00.981670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:00.982725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:00.983694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:00.983694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:00.983694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:00.983694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:00.983694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 599/1203] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:00.989516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:01.089531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-352386e9-20fc-4e36-87f8-85707c57f576 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:01.118127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:01.123651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:01.124346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:01.124688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:01.129342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:01.129342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:01.129342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:01.129342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf04710f-7906-45c2-8f5d-b2a2e130a185 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:01.138511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 605/1204] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:01.144269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:01.197571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:01.243972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d935c2-da58-43b1-9c24-865e845437c2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:01.277024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c267b08b-226c-4156-a8d6-e04a2fba142e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:01.277777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267b08b-226c-4156-a8d6-e04a2fba142e None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:01.278067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267b08b-226c-4156-a8d6-e04a2fba142e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:01.278345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c267b08b-226c-4156-a8d6-e04a2fba142e None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:01.279044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267b08b-226c-4156-a8d6-e04a2fba142e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:01.280502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c267b08b-226c-4156-a8d6-e04a2fba142e None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:01.419609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 600/1205] 10.222.0.40 () {60 vars in 1132 bytes} [Thu Jun 16 11:41:01 2022] POST /identity/v3/users => generated 251 bytes in 276 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:41:03.543543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90185fee-38b4-4c1e-84f1-92aa74f04c5c None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:03.544154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90185fee-38b4-4c1e-84f1-92aa74f04c5c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:03.544255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90185fee-38b4-4c1e-84f1-92aa74f04c5c None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:03.545572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 606/1206] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:03.550719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:03.561043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:03.561043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:03.569280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:03.570457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:03.635496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 601/1207] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:03.659810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90185fee-38b4-4c1e-84f1-92aa74f04c5c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:03.759794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90185fee-38b4-4c1e-84f1-92aa74f04c5c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:03.792811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:03.793644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:03.793898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:03.794157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:03.804103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:41:03.811732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:03.812941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:03.813789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 607/1208] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:41:03 2022] GET /identity/v3/roles/service => generated 85 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:03.819079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:03.858442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:03.915626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8175931a-97cb-4b62-99a4-8b4bf293534b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:03.947736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:03.948515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:03.948797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:03.949071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:03.949912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:03.950184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:03.951993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:03.960560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 602/1209] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:41:03 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 142 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:03.966714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:04.015738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:04.062912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b040391-8989-4b9a-b6cc-b928cb4720db None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:04.101814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:04.102598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:04.102884 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:04.103185 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:04.114328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:04.115802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:04.132693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:04.133709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:04.134761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:04.134761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:04.134761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:04.134761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:04.134761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 608/1210] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:03 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:41:04.139657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:04.234525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:04.237433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9de5f9-84fe-42f5-85da-1c5bc116063a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:04.270364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:04.271020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:04.271304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:04.271579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:04.272339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:04.272646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:04.274002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:04.285409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 603/1211] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:04.291027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:04.401671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-892c6716-b84c-478f-bda8-e8560a26234b None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:04.403101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:04.434161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:04.434953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:04.435234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:04.435508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:04.443647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:04.446924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:41:04.459737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:04.460851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:04.462375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 609/1212] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:04 2022] GET /identity/v3/users/placement => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:04.467139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:04.564530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5123a079-34f4-4cd4-8882-33183b622a36 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:04.597163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:04.597838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:04.598124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:04.598431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:04.599261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:04.599574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:04.601034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:04.617945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 604/1213] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 11:41:04 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:04.623696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:04.723280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f28112d-ff32-473c-a12c-79a4bf0b9631 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:04.758600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:04.759243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:04.759526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:04.759799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:04.789264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:04.824809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:04.826239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:04.922033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 610/1214] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:41:04 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 299 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:41:04.927838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:04.946108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:05.024892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71795c7e-b664-4c70-a202-3959fc24237a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:05.057464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:05.058248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:05.058532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:05.058808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:05.068409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:05.069826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:05.088140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:05.089098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:05.090504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:05.090504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:05.090504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:05.090504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:05.090504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 605/1215] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:04 2022] GET /identity/v3/domains/Default => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:05.095563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:05.154861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:05.192086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52f3725-47c2-4960-bf8c-d87b5c217c58 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:05.226209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:05.226873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:05.227160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:05.227441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:05.228194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:05.228510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:05.229872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:05.241476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 611/1216] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:05.249132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:05.315121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:05.345314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7137b29-b51f-40f2-9a27-b20641622ec2 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:05.378655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:05.379435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:05.379721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:05.380030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:05.386465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:05.387920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:41:05.402072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:05.403180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:05.404445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 606/1217] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:41:05 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:05.409274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:05.514724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dcdf3ac-3ad7-4337-86e7-6aac6789b514 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:05.536728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:05.547665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:05.548402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:05.548831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:05.549112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:05.549888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:05.550198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:05.551610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ebb38d40-33fd-4bd5-a884-f5a67eff56ca None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:05.563693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 612/1218] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:41:05 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:05.569659 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:05.667821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a44a4ea-37b7-4984-826e-4788ce409911 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:05.696615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:05.702342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:05.703540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:05.703822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:05.704126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:05.704985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=f482060b5ab3458ba78d94184d68f10d) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:05.705301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:05.706711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abf4072a-61e7-40ca-ae18-221053022921 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:05.712305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 607/1219] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:41:05 2022] GET /identity/v3/role_assignments?user.id=f482060b5ab3458ba78d94184d68f10d&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:07.802872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd68820-7325-4591-a0b9-b09858c654a8 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:07.803463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd68820-7325-4591-a0b9-b09858c654a8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:07.803463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dd68820-7325-4591-a0b9-b09858c654a8 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:07.804828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 613/1220] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:07.810192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:07.810594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:07.810754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:07.827183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:07.833428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:07.893535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 608/1221] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:07 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:07.915983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd68820-7325-4591-a0b9-b09858c654a8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.016722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dd68820-7325-4591-a0b9-b09858c654a8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:08.049921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:08.053780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:08.053780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:08.053780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] PATH_INFO: `/v3/roles/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:08.061356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:41:08.069189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:08.070483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:08.071774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 614/1222] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:41:07 2022] GET /identity/v3/roles/service => generated 85 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:08.076668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.173417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5dc28d8-c167-4230-9efa-a93aee7fb72f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:08.196374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:08.207134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:08.207794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:08.208107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:08.208384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:08.209126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] RBAC: Adding query filter params (name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:08.209430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:08.210777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:08.219190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 609/1223] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:41:08 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:08.224998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.268152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:08.321774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a09d5019-f731-4ba7-b981-0321576c2427 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:08.354487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:08.355283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:08.355589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:08.356090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:08.367672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:08.369141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:08.386196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:08.387225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:08.388253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:08.388253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:08.388253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:08.388253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:08.388253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 615/1224] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:08.393914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.489468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0f85db3-7ec9-464a-a67e-ddc10c564f44 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:08.524485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:08.524485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:08.524485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:08.524485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:08.524899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:08.525129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:08.526485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:08.542880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 610/1225] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:08 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:08.548674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.642948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:08.649837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e9ed38-5794-449e-8355-1646af6d9311 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:08.683641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:08.684483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:08.684769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:08.685069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] PATH_INFO: `/v3/users/placement` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:08.692937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:08.694375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:41:08.709157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:08.710288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:08.711513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 616/1226] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:08 2022] GET /identity/v3/users/placement => generated 87 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:08.716573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.763003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:08.817976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7cf70fd-80c1-4f8a-b2ef-0e0059f50bb8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:08.850131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:08.850789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:08.851072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:08.851346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:08.852201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:08.852513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:08.853978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:08.870668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 611/1227] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 11:41:08 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:08.876853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:08.975233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ddbc79e-9efe-4d89-bc02-224d43e3573f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:09.010235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:09.011028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:09.011316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:09.011591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:09.021538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:09.022966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:09.040173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:09.049839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:09.050949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:09.050949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:09.050949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:09.050949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:09.050949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 617/1228] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:08 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:09.051308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:09.144923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e9d57a-f98b-4738-ad24-054fb3c919ce None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:09.147088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:09.177381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:09.178048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:09.178331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:09.178602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:09.179309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:09.179613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:09.180999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:09.192029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 612/1229] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:09.197854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:09.206629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:09.294296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17f7e77e-e4a9-4643-b58d-552dbaa0d0a5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:09.328264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:09.329059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:09.329351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:09.329633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:09.336135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:09.337823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:41:09.351862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:09.352948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:09.354201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 618/1230] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:41:09 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:09.358790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:09.455891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3372df2-f335-4512-a1f1-b6c2fef3455f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:09.489252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:09.489914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:09.490202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:09.490481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:09.491260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:09.491573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:09.493083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc724803-4562-429f-bfc2-4dc5afec8272 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:09.504635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 613/1231] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:41:09 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:09.510640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:09.575362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:09.610898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-faca028a-7576-4721-a1dd-619d17a78600 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:09.643535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55eba4ac-a721-458d-b45f-8739d6a292b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:09.644450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba4ac-a721-458d-b45f-8739d6a292b1 None admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:09.644736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba4ac-a721-458d-b45f-8739d6a292b1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:09.645013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55eba4ac-a721-458d-b45f-8739d6a292b1 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/f482060b5ab3458ba78d94184d68f10d/roles/7d569182856e455cb3b951a1d1163f58` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:09.660001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eba4ac-a721-458d-b45f-8739d6a292b1 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=f482060b5ab3458ba78d94184d68f10d, role_id=7d569182856e455cb3b951a1d1163f58)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:09.661482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55eba4ac-a721-458d-b45f-8739d6a292b1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:09.768833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 619/1232] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:41:09 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/f482060b5ab3458ba78d94184d68f10d/roles/7d569182856e455cb3b951a1d1163f58 => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:41:11.844561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475f90d-5cdc-4491-b540-939030cefbbe None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:11.845109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475f90d-5cdc-4491-b540-939030cefbbe None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:11.845213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0475f90d-5cdc-4491-b540-939030cefbbe None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:11.846558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 614/1233] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:11.851563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:11.852025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:11.852161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:11.868472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:11.869684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:11.937276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 620/1234] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:11.958933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475f90d-5cdc-4491-b540-939030cefbbe None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:11.983388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:12.063210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0475f90d-5cdc-4491-b540-939030cefbbe None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:12.097175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:12.097969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:12.098381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:12.098666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] PATH_INFO: `/v3/roles/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:12.108835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:41:12.117405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:12.118577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:12.119449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 615/1235] 10.222.0.40 () {56 vars in 1102 bytes} [Thu Jun 16 11:41:11 2022] GET /identity/v3/roles/service => generated 85 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:12.124844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:12.170297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:12.222407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bd1510c-2929-4e2f-87ce-74dd2aa05a91 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:12.255331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:12.256032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:12.256323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:12.256603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:12.257427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] RBAC: Adding query filter params (name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:12.257938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:12.259904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:12.267570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 621/1236] 10.222.0.40 () {56 vars in 1103 bytes} [Thu Jun 16 11:41:12 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:12.273352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:12.358699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:12.374254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f523e1f-09d1-4774-aa0d-c03ecac7aa7e None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:12.406155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:12.406944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:12.407720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:12.408034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:12.418843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:12.420306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:12.437929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:12.438991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:12.440082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:12.440082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:12.440082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:12.440082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:12.440082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 616/1237] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:12.444865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:12.541583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29d701cf-8c1b-4be5-b2a3-f50632a69fde None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:12.549999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:12.573358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:12.574011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:12.574293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:12.574563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:12.575276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:12.575582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:12.576952 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:12.589475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 622/1238] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:12.595770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:12.694200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd661aaf-5a59-4e6b-bfb8-14e5e0f0945f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:12.728066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:12.728861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:12.729142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:12.729415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:12.732139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:12.737056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:12.741514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:41:12.752441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:12.753655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:12.755022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 617/1239] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:12 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:12.759447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:12.858525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b06faef-0ef4-4243-99c4-bd350c8bd979 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:12.891378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:12.892784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:12.893062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:12.893458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:12.894166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:12.894484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:12.895926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:12.912397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 623/1240] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 11:41:12 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:12.918203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:13.016257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d18b94-e074-4c1a-997d-b10c4a545e98 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:13.049196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:13.049837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:13.050119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:13.050394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:13.109932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:13.116330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:13.117773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:13.211692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 618/1241] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:41:12 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:41:13.218010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:13.245058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:13.315732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b269c9-3310-4045-921b-a78bd5c35fdc None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:13.349345 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:13.350127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:13.350408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:13.350679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:13.360902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:13.362310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:13.379733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:13.380824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:13.382224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:13.382224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:13.382224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:13.382224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:13.382224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 624/1242] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:13 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:13.386568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:13.478727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:13.484340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54af859-f462-40b1-bbc6-263de973fe55 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:13.517179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:13.517851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:13.518137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:13.518508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:13.519227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:13.519537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:13.520929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:13.531786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 619/1243] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:13.537481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:13.612309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:13.632095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9780ee2c-3a51-487c-a2c5-a9252d24adeb None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:13.664037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:13.664814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:13.665096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:13.665372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:13.671725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:13.673779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:41:13.688643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:13.695672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:13.696813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 625/1244] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:41:13 2022] GET /identity/v3/projects/service => generated 88 bytes in 153 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:13.697400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:13.792421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5bd4339-fb9c-441d-a4ef-b4d838321f13 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:13.825082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:13.825924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:13.826242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:13.826526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:13.827383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:13.827694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:13.829166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b62430f-2a61-4a01-a41c-c28bcbdf70ad None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:13.841536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 620/1245] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:41:13 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:13.848093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:13.946022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20eea8c5-b2c5-403a-9ab0-b9561f46208e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:13.979088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:13.979747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:13.980060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:13.980342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:13.981176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] RBAC: Adding query filter params (role.id=7d569182856e455cb3b951a1d1163f58, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=f482060b5ab3458ba78d94184d68f10d) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:13.981492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:13.982918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60bd159e-cdb8-4ce4-804b-983b3ef1cebd None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:13.988754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 626/1246] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:41:13 2022] GET /identity/v3/role_assignments?user.id=f482060b5ab3458ba78d94184d68f10d&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=7d569182856e455cb3b951a1d1163f58 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:16.135304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ca75-de06-4544-aef7-63677dbd0f59 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:16.135860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ca75-de06-4544-aef7-63677dbd0f59 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:16.135860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6260ca75-de06-4544-aef7-63677dbd0f59 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:16.137306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 621/1247] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:16.143068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:16.143477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:16.143623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:16.148874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:16.160739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:16.161904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:16.227737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 627/1248] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:16 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:16.251604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ca75-de06-4544-aef7-63677dbd0f59 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:16.269095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:16.353648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6260ca75-de06-4544-aef7-63677dbd0f59 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:16.388563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:16.389494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:16.389915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:16.390339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:16.400417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:41:16.408182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:16.414737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:16.415813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 622/1249] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:41:16 2022] GET /identity/v3/roles/admin => generated 83 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:16.416703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:16.511959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1a3830-5775-451a-9892-eee3dcaf8918 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:16.540397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:16.545244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:16.545911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:16.546199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:16.546477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:16.547221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:16.547545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:16.549442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:16.557437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 628/1250] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:41:16 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:41:16.564112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:16.639005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:16.661219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15e989f1-0da6-4894-9e6f-c76e83b9edc0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:16.694814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:16.695608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:16.695919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:16.696200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:16.706756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:16.708210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:16.725363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:16.726413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:16.727506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:16.727506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:16.727506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:16.727506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:16.727506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 623/1251] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:16.732440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:16.828323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-902306bc-611a-4b9f-8cf7-d1ec6ad146c1 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:16.863336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:16.864027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:16.864338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:16.864616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:16.865328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:16.865635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:16.866989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:16.879073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 629/1252] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:16.884916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:16.986513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c4c25f2-0ca7-49da-843f-5cf53604735a None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:17.012869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:17.021072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:17.021875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:17.022162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:17.022437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:17.030315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:17.032725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:41:17.046875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:17.048024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:17.049162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 624/1253] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:16 2022] GET /identity/v3/users/placement => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:17.053933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:17.108657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:17.153920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24902c4c-9e50-4508-85f1-4ba5e6ff8c83 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:17.187318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:17.188011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:17.188327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:17.188607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:17.189521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:17.189835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:17.191259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:17.208251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 630/1254] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 11:41:17 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:17.214297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:17.311974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c217a8a-00d9-4cac-8227-12e6416e49b5 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:17.345318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:17.345956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:17.346282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:17.346557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:17.402122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:17.413985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:17.415426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:17.510660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 625/1255] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:41:17 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 297 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:41:17.517097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:17.614192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c48ad1a0-e219-42e9-9b8d-e19b87607b0c None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:17.627287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:17.649086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:17.649877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:17.650164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:17.650439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:17.660383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:17.661829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:17.679213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:17.680397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:17.682442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:17.682442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:17.682442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:17.682442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:17.682442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 631/1256] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:17 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:17.686428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:17.772819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:17.784346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c232993d-08dd-45e1-9d87-1ab235cf36e6 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:17.818198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:17.818534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:17.818818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:17.819098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:17.819816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:17.820152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:17.821523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:17.832986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 626/1257] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:17.838994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:17.940102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6c73716-76e2-4f4b-b9d2-873f8236e003 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:17.973727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:17.974517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:17.974805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:17.975092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:17.981707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:17.983435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:41:17.997518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:17.998535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:17.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 632/1258] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:41:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:18.005361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:18.105128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad2c45b-f80f-42c3-be68-511d6a982ea8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:18.138375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:18.139036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:18.139327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:18.139607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:18.140422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:18.140738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:18.142416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-440f354d-e455-4d93-a59e-90a290bc74f5 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:18.154351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 627/1259] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:41:18 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:18.161103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:18.186893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:18.260344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3329e426-0fce-4b56-abe3-93057fe48b74 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:18.293738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:18.294390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:18.294679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:18.294962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:18.295790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=f482060b5ab3458ba78d94184d68f10d) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:18.296136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:18.297564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-172dfbb0-0ecf-4f5b-b2c6-4176d1e87afb None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:18.303917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 633/1260] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:41:18 2022] GET /identity/v3/role_assignments?user.id=f482060b5ab3458ba78d94184d68f10d&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=fda22db004a64c87b35510b321190ad4 => generated 259 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:20.377786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083e3e4-2990-43fb-b305-5d36a43e662a None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:20.378357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083e3e4-2990-43fb-b305-5d36a43e662a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:20.378357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0083e3e4-2990-43fb-b305-5d36a43e662a None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:20.379690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 628/1261] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:20.384925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:20.385584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:20.385858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:20.403019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:20.404243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:20.469086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 634/1262] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:20 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:41:20.491790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083e3e4-2990-43fb-b305-5d36a43e662a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:20.502595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:20.592293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0083e3e4-2990-43fb-b305-5d36a43e662a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:20.626255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:20.627046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:20.627333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:20.627610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:20.637524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:41:20.645187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:20.646376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:20.647552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 629/1263] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:41:20 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:20.652659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:20.670978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:20.751859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43d07d6c-6235-4bfe-be8c-343f18904a5b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:20.784828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:20.785489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:20.785770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:20.786042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:20.786779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:20.787083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:20.788506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:20.796260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 635/1264] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:41:20 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:20.801910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:20.867537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:20.899755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b141b042-ef6f-4993-9f88-218f5b5b4532 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:20.933699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:20.934501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:20.934792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:20.935075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:20.946769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:20.948248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:20.965752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:20.968948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:20.970461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:20.970461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:20.970461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:20.970461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:20.970461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 630/1265] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:20.973280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:21.057867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:21.076941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54711e27-42a8-48bc-a4b7-cb231ce32b64 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:21.114377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:21.115150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:21.115440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:21.115719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:21.116468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:21.116779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:21.118276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:21.130250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 636/1266] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:21.136107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:21.231732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14272f96-cf38-4c28-bfc3-0803b4d1b0df None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:21.250110 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:21.264004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:21.264794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:21.265076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:21.265348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] PATH_INFO: `/v3/users/placement` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:21.273041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:21.274457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:41:21.288278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:21.289368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:21.290530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 631/1267] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:21 2022] GET /identity/v3/users/placement => generated 87 bytes in 154 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:21.295312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:21.395975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4ff5799-afd4-4c28-a048-564c5b97d092 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:21.428712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:21.429421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:21.429707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:21.429986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:21.430813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:21.431128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:21.432568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:21.442130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:21.450459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 637/1268] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 11:41:21 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:21.456793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:21.555571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f43591-99f2-41d7-adf5-4d57a98536c0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:21.588289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:21.589239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:21.589524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:21.589795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:21.600166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:21.601598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:21.618753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:21.619790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:21.620899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:21.620899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:21.620899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:21.620899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:21.620899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 632/1269] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:21.626467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:21.725169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1833649f-a160-4ee7-a07b-d7c6a8121921 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:21.758979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:21.759643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:21.759962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:21.760245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:21.760960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:21.761270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:21.762632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:21.773484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 638/1270] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:21.779343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:21.810717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:21.878733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dd22acc-f2a6-4c20-9c72-abe1f28b0599 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:21.911514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:21.912337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:21.912623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:21.912899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:21.919471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:21.920961 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:41:21.935571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:21.936801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:21.938070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 633/1271] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:41:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:21.942591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:22.000225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:22.039829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c62704cd-4190-40b0-815f-0eeacc7ca3f6 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:22.073054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02646f-d762-411a-922b-72c636332fec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:22.073652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02646f-d762-411a-922b-72c636332fec None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:22.073937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02646f-d762-411a-922b-72c636332fec None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:22.074212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02646f-d762-411a-922b-72c636332fec None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:22.075040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02646f-d762-411a-922b-72c636332fec None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:22.075340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02646f-d762-411a-922b-72c636332fec None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:22.076786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02646f-d762-411a-922b-72c636332fec None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:22.089056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 639/1272] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:41:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:22.095592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:22.186008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:22.194871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8c528cd-271e-4550-897d-63a5b4f3cb9f None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:22.227969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c5d976a-75ce-4032-a7cf-e07f0527da95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:22.228914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d976a-75ce-4032-a7cf-e07f0527da95 None admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:22.229175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d976a-75ce-4032-a7cf-e07f0527da95 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:22.229451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c5d976a-75ce-4032-a7cf-e07f0527da95 None admin] PATH_INFO: `/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/f482060b5ab3458ba78d94184d68f10d/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:22.231402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d976a-75ce-4032-a7cf-e07f0527da95 None admin] RBAC: Authorizing `identity:create_grant(project_id=4e5224f2718d4d228b36c920cab06d42, user_id=f482060b5ab3458ba78d94184d68f10d, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:22.232864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c5d976a-75ce-4032-a7cf-e07f0527da95 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:22.345370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 634/1273] 10.222.0.40 () {58 vars in 1439 bytes} [Thu Jun 16 11:41:22 2022] PUT /identity/v3/projects/4e5224f2718d4d228b36c920cab06d42/users/f482060b5ab3458ba78d94184d68f10d/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:41:24.442910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f135f-a36d-4535-a6d3-e7a5ad2f3864 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:24.443462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f135f-a36d-4535-a6d3-e7a5ad2f3864 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:24.443462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03f135f-a36d-4535-a6d3-e7a5ad2f3864 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:24.444854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 640/1274] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:24.449958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:24.450396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:24.450539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:24.467178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:24.468386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:24.533988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 635/1275] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jun 16 11:41:24.556889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f135f-a36d-4535-a6d3-e7a5ad2f3864 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:24.658434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03f135f-a36d-4535-a6d3-e7a5ad2f3864 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:24.674926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:24.691217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:24.692040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:24.692326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:24.692603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:24.702419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(role_id) Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jun 16 11:41:24.710474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:24.711551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:24.712906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 641/1276] 10.222.0.40 () {56 vars in 1096 bytes} [Thu Jun 16 11:41:24 2022] GET /identity/v3/roles/admin => generated 83 bytes in 155 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:24.717963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:24.797827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:24.814970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6051a45-77e4-4c43-b955-cc063e94bcd9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:24.849693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:24.850362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:24.850649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:24.850928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:24.851749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] RBAC: Adding query filter params (name=admin) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:24.852080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:24.853444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:24.861273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 636/1277] 10.222.0.40 () {56 vars in 1099 bytes} [Thu Jun 16 11:41:24 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:24.867007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:24.965928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a593bed-b6dd-4ca9-9fea-7f8f6c989dc5 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:24.997970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:24.998753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:24.999038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:24.999316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:25.010158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:25.011600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:25.029040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:25.030091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:25.031269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:25.031269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:25.031269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:25.031269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:25.031269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 642/1278] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:25.036041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:25.132025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75134c53-b265-4c36-aa72-7e60d25d7b33 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:25.165748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:25.166452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:25.166731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:25.167006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:25.167717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:25.168085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:25.169445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:25.180878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 637/1279] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:25.186767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:25.217937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:25.285522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e36871c0-f6f8-4356-b774-28f6b85d482e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:25.319457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:25.320285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:25.320575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:25.320856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] PATH_INFO: `/v3/users/placement` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:25.327791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:25.329257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_user(user_id) Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jun 16 11:41:25.343995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:25.345086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:25.346031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 643/1280] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:25 2022] GET /identity/v3/users/placement => generated 87 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jun 16 11:41:25.350954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:25.354718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:25.447297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f46c26-a85f-486c-bf58-ba7176946921 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:25.481297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:25.481927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:25.482210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:25.482485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:25.483307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:25.483616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:25.485577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:25.502497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 638/1281] 10.222.0.40 () {56 vars in 1143 bytes} [Thu Jun 16 11:41:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:25.508174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:25.596258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:25.605765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99c1be28-1a9e-404f-a83c-55936d74ab4f None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:25.638559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:25.639204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:25.639491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:25.639765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:25.707096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:25.709902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:25.793586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:25.803463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 644/1282] 10.222.0.40 () {58 vars in 1282 bytes} [Thu Jun 16 11:41:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 296 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jun 16 11:41:25.809141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:25.869180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:25.905827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04053ff4-497d-4bae-89b4-8491c319e8a0 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:25.937480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:25.938797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:25.939080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:25.939353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:25.949060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:25.950908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jun 16 11:41:25.970446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:25.971467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:25.972789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jun 16 11:41:25.972789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jun 16 11:41:25.972789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jun 16 11:41:25.972789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:25.972789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 639/1283] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:41:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:25.977195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:26.072326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aa08f0c-5e9e-42f3-9abb-2de98d711612 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:26.105377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:26.106071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:26.106375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:26.106638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:26.107361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:26.107673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:26.109086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:26.120564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 645/1284] 10.222.0.40 () {56 vars in 1109 bytes} [Thu Jun 16 11:41:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:26.126930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:26.223570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6dfba18-264d-4467-ac00-3aa3347685a8 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:26.237734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:26.257972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:26.258761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:26.259046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:26.259322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] PATH_INFO: `/v3/projects/service` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:26.265760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:26.267222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(project_id) Jun 16 11:41:26.281306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:26.282339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: ERROR keystone.server.flask.application Jun 16 11:41:26.283466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 640/1285] 10.222.0.40 () {56 vars in 1111 bytes} [Thu Jun 16 11:41:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:26.288622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:26.338478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:26.386831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e114fc3e-1e66-4a93-9874-0369550e8ca7 None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:26.419242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:26.419925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:26.420223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:26.420492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:26.421269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:26.421581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:26.423635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3280804f-f1d4-4ebe-b1ce-e1008a029b12 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:26.435327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 646/1286] 10.222.0.40 () {56 vars in 1148 bytes} [Thu Jun 16 11:41:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:26.441393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:26.538330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c73c62e8-505c-4f5f-85a3-c21e7905ea85 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:26.571347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:26.572029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:26.572317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:26.572597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:26.573426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] RBAC: Adding query filter params (role.id=fda22db004a64c87b35510b321190ad4, scope.project.id=4e5224f2718d4d228b36c920cab06d42, user.id=f482060b5ab3458ba78d94184d68f10d) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:26.573742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:26.575166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29c6d6f0-e6de-4038-8d4d-50778fbff0c1 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:26.580869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 641/1287] 10.222.0.40 () {56 vars in 1374 bytes} [Thu Jun 16 11:41:26 2022] GET /identity/v3/role_assignments?user.id=f482060b5ab3458ba78d94184d68f10d&scope.project.id=4e5224f2718d4d228b36c920cab06d42&role.id=fda22db004a64c87b35510b321190ad4 => generated 608 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:28.720122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b3923-9f00-4845-b01b-5baf72f68292 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:28.720645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b3923-9f00-4845-b01b-5baf72f68292 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:28.720751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-882b3923-9f00-4845-b01b-5baf72f68292 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:28.722083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 647/1288] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:28.727128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:28.727553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:28.727693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:28.744232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:28.745449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:28.774902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:28.809881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 642/1289] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:28.833240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b3923-9f00-4845-b01b-5baf72f68292 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:28.930834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-882b3923-9f00-4845-b01b-5baf72f68292 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:28.941786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:28.965102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:28.965897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:28.966183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:28.966461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] PATH_INFO: `/v3/services/placement` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:28.967141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:28.968528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:28.981127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:28.982788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:28.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 648/1290] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:41:28 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:28.988799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:29.085596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5330948f-da89-41d9-a300-1e2896a25757 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:29.118888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:29.119551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:29.119835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:29.120156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:29.120938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:29.121247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:29.122596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ee3490-1ce6-4e2d-9f83-e3b3b0b9d928 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:29.128132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 643/1291] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 11:41:28 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:41:29.133960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:29.232182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94072e5b-3908-4b69-9439-c5669cceadac None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:29.265253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:29.265936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:29.266225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:29.266505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] PATH_INFO: `/v3/services` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:29.267208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] RBAC: Adding query filter params (type=placement) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:29.267519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:29.268924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb8d02d6-8101-41b7-8fbc-e67c2a48b89c None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:29.273251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 649/1292] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 11:41:29 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:31.359895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4621fb-3458-4dff-8acd-f09b682bd8d9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:31.360423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4621fb-3458-4dff-8acd-f09b682bd8d9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:31.360540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d4621fb-3458-4dff-8acd-f09b682bd8d9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:31.361822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 644/1293] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:31.367093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842896b-7574-4a44-b658-21dd70196c57 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:31.367499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842896b-7574-4a44-b658-21dd70196c57 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:31.367650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e842896b-7574-4a44-b658-21dd70196c57 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:31.384164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e842896b-7574-4a44-b658-21dd70196c57 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:31.385346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e842896b-7574-4a44-b658-21dd70196c57 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:31.414749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:31.450255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 650/1294] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:31 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:31.474067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4621fb-3458-4dff-8acd-f09b682bd8d9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:31.504797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:31.574194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d4621fb-3458-4dff-8acd-f09b682bd8d9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:31.612403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c4498-2e85-4fdc-b5e5-d5633586e8bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:31.613007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c4498-2e85-4fdc-b5e5-d5633586e8bd None admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:31.613294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c4498-2e85-4fdc-b5e5-d5633586e8bd None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:31.613575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c4498-2e85-4fdc-b5e5-d5633586e8bd None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:31.614292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c4498-2e85-4fdc-b5e5-d5633586e8bd None admin] RBAC: Authorizing `identity:create_service()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:31.615659 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c4498-2e85-4fdc-b5e5-d5633586e8bd None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:31.693599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 645/1295] 10.222.0.40 () {60 vars in 1141 bytes} [Thu Jun 16 11:41:31 2022] POST /identity/v3/services => generated 246 bytes in 220 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:41:33.818703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bb84c-188d-4821-86d5-2aa2df7549ae None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:33.819217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bb84c-188d-4821-86d5-2aa2df7549ae None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:33.819308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b07bb84c-188d-4821-86d5-2aa2df7549ae None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:33.820675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 651/1296] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:33.825696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:33.826147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:33.826284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:33.843546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:33.845243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:33.914881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 646/1297] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:33 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:33.938320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bb84c-188d-4821-86d5-2aa2df7549ae None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:34.037740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b07bb84c-188d-4821-86d5-2aa2df7549ae None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:34.070420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:34.071218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:34.071501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:34.071776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] PATH_INFO: `/v3/services/placement` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:34.072459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:34.074030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:34.086238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:34.087672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:34.088631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 652/1298] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:41:33 2022] GET /identity/v3/services/placement => generated 90 bytes in 150 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:34.093860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:34.111555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:34.189321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d560c0-5c13-45ed-846f-d95417392252 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:34.221724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:34.222380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:34.222661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:34.222938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:34.223640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:34.231348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:34.231348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:34.231348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 647/1299] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 11:41:34 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:34.237035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:34.268981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:34.334939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6a5737d-aa8d-4272-8e5b-94b5f4e93bad None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:34.368441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:34.369095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:34.369381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:34.369663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:34.370444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] RBAC: Adding query filter params (interface=public, service_id=fb68ba0eec744f55b94c01345f205693, region_id=RegionOne) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:34.370757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:34.372140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8bdd670-cac3-4818-a6a7-5da933b7f681 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:34.377596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 653/1300] 10.222.0.40 () {56 vars in 1251 bytes} [Thu Jun 16 11:41:34 2022] GET /identity/v3/endpoints?service_id=fb68ba0eec744f55b94c01345f205693&interface=public®ion_id=RegionOne => generated 194 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:34.383439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:34.462964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:34.479947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f8d9c2d-da7c-4fb0-b26a-e52d4f056c90 None admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:34.513781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-663a0810-63db-4af1-a784-9cd662eff6df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:34.514462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0810-63db-4af1-a784-9cd662eff6df None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:34.514729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0810-63db-4af1-a784-9cd662eff6df None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:34.515005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-663a0810-63db-4af1-a784-9cd662eff6df None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:34.515708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a0810-63db-4af1-a784-9cd662eff6df None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:34.517103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-663a0810-63db-4af1-a784-9cd662eff6df None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:34.523801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 648/1301] 10.222.0.40 () {56 vars in 1087 bytes} [Thu Jun 16 11:41:34 2022] GET /identity/v3/services => generated 2191 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:41:36.631073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbe861-0078-451e-b02a-370d8a12d63f None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:36.631628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbe861-0078-451e-b02a-370d8a12d63f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:36.631628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70cbe861-0078-451e-b02a-370d8a12d63f None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:36.633031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 654/1302] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:41:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:41:36.638122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:36.638521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:36.638698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:36.655229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:41:36.657085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:41:36.726776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 649/1303] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:41:36 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jun 16 11:41:36.749155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbe861-0078-451e-b02a-370d8a12d63f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:36.850302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:36.857102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70cbe861-0078-451e-b02a-370d8a12d63f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:36.891718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:36.892473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:36.892755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:36.893103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] PATH_INFO: `/v3/services/placement` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:36.893756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:36.895115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Traceback (most recent call last): Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jun 16 11:41:36.907367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._get_service(service_id) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jun 16 11:41:36.908555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.get_or_create( Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application with Lock( Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self._enter() Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application return self.creator() Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application created_value = creator( Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: ERROR keystone.server.flask.application Jun 16 11:41:36.909608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 655/1304] 10.222.0.40 () {56 vars in 1117 bytes} [Thu Jun 16 11:41:36 2022] GET /identity/v3/services/placement => generated 90 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jun 16 11:41:36.915141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:36.992633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:37.016693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d9eab32-beb6-4460-83c3-aee9adfd5a32 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:37.050037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:37.050752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:37.051047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:37.051333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] PATH_INFO: `/v3/services` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:37.052083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] RBAC: Adding query filter params (name=placement) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:41:37.052403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:37.053773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fd46f2f-ff5f-437e-8fae-658765d380d3 None admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:37.059106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 650/1305] 10.222.0.40 () {56 vars in 1116 bytes} [Thu Jun 16 11:41:36 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:41:37.065328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:41:37.166771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3a951eb-9166-4c85-b420-a0093899044e None admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:41:37.199474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b0c6dea-0f53-406c-9230-0d50019bdd83 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:41:37.200143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6dea-0f53-406c-9230-0d50019bdd83 None admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:41:37.200430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6dea-0f53-406c-9230-0d50019bdd83 None admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:41:37.200705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b0c6dea-0f53-406c-9230-0d50019bdd83 None admin] PATH_INFO: `/v3/endpoints` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:41:37.201431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c6dea-0f53-406c-9230-0d50019bdd83 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:41:37.202788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b0c6dea-0f53-406c-9230-0d50019bdd83 None admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:41:37.299105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 656/1306] 10.222.0.40 () {60 vars in 1144 bytes} [Thu Jun 16 11:41:37 2022] POST /identity/v3/endpoints => generated 331 bytes in 234 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:41:47.060327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=30 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:41:47.294043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:45.563407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded999b7-577b-49ec-afe4-8f7b85acb5e3 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:45.563959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded999b7-577b-49ec-afe4-8f7b85acb5e3 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:45.563959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded999b7-577b-49ec-afe4-8f7b85acb5e3 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:45.570445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc07de5-dd25-44b9-b166-c08f6bb7aafd None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:45.570902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc07de5-dd25-44b9-b166-c08f6bb7aafd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:45.570985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdc07de5-dd25-44b9-b166-c08f6bb7aafd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:45.574408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 651/1307] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:47:45 2022] GET /identity => generated 270 bytes in 12 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:45.661702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-bdc07de5-dd25-44b9-b166-c08f6bb7aafd None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:45.662949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bdc07de5-dd25-44b9-b166-c08f6bb7aafd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:45.843992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 657/1308] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:47:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 273 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:47:45.890043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363596-ab99-4a45-b05b-b3c1bf14ef0d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:45.890829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363596-ab99-4a45-b05b-b3c1bf14ef0d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:45.891160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2363596-ab99-4a45-b05b-b3c1bf14ef0d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:45.892592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 652/1309] 10.222.0.40 () {54 vars in 964 bytes} [Thu Jun 16 11:47:45 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:45.898849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33b32-0214-4e8f-af8a-0b70d725f9dc None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:45.899212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33b32-0214-4e8f-af8a-0b70d725f9dc None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:45.899568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda33b32-0214-4e8f-af8a-0b70d725f9dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:45.980865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-dda33b32-0214-4e8f-af8a-0b70d725f9dc None None] MFA Rules not processed for user `516b221cb1794430befadcde5f9fd77b`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:45.982258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-dda33b32-0214-4e8f-af8a-0b70d725f9dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:46.004103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:46.064692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 658/1310] 10.222.0.40 () {58 vars in 1061 bytes} [Thu Jun 16 11:47:45 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:47:46.074089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2363596-ab99-4a45-b05b-b3c1bf14ef0d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:47:46.075937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b2363596-ab99-4a45-b05b-b3c1bf14ef0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:46.193859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2363596-ab99-4a45-b05b-b3c1bf14ef0d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:47:46.274620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:47:46.275282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:46.275568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:46.275846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:46.278035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:46.334480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:46.390443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:47:46.391912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb89464-7e39-42b3-a082-cff5cae3d810 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:47:46.411990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 653/1311] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:47:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:47:48.823660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad712e-e267-473f-b674-01d80aedef59 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:48.824249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad712e-e267-473f-b674-01d80aedef59 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:48.824249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfad712e-e267-473f-b674-01d80aedef59 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:48.825803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 659/1312] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:47:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:48.830772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f702e9-37d3-4232-9a39-239c7a2c6e1a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:48.831242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f702e9-37d3-4232-9a39-239c7a2c6e1a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:48.831324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29f702e9-37d3-4232-9a39-239c7a2c6e1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:48.903122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-29f702e9-37d3-4232-9a39-239c7a2c6e1a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:48.904369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-29f702e9-37d3-4232-9a39-239c7a2c6e1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:49.007911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 654/1313] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:47:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:47:49.044027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad712e-e267-473f-b674-01d80aedef59 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:47:49.045730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-dfad712e-e267-473f-b674-01d80aedef59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:49.166621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfad712e-e267-473f-b674-01d80aedef59 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:47:49.248713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:49.251500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:47:49.252274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:49.252646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:49.252998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:49.254734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:49.372036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:47:49.373727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f3b065e-242b-40d7-91c4-aee426f852fb service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:47:49.394566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 660/1314] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:47:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 351 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:47:51.946706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedcb6c-d35e-4b01-8010-dcd63d1f01d8 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:51.947212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedcb6c-d35e-4b01-8010-dcd63d1f01d8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:51.947212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bedcb6c-d35e-4b01-8010-dcd63d1f01d8 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:51.948618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 655/1315] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:47:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:51.953712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ebcd9-8386-4efb-8004-ded38f058421 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:51.954129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ebcd9-8386-4efb-8004-ded38f058421 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:51.954290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17ebcd9-8386-4efb-8004-ded38f058421 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:52.017243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a17ebcd9-8386-4efb-8004-ded38f058421 None None] MFA Rules not processed for user `05e24c3335b843adafa09ce1a2c81793`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:52.018427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a17ebcd9-8386-4efb-8004-ded38f058421 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:52.069970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:52.112703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 661/1316] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:47:51 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:47:52.166422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22baeb-c8de-46da-a172-473fe85095c1 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:52.167350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22baeb-c8de-46da-a172-473fe85095c1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:52.167632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e22baeb-c8de-46da-a172-473fe85095c1 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:52.169045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 656/1317] 10.222.0.40 () {54 vars in 964 bytes} [Thu Jun 16 11:47:52 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:52.175042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b5bee-d01a-4ca0-9971-37ae4308cb30 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:52.175392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b5bee-d01a-4ca0-9971-37ae4308cb30 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:52.180932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-523b5bee-d01a-4ca0-9971-37ae4308cb30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:52.254391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-523b5bee-d01a-4ca0-9971-37ae4308cb30 None None] MFA Rules not processed for user `516b221cb1794430befadcde5f9fd77b`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:52.255600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-523b5bee-d01a-4ca0-9971-37ae4308cb30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:52.328702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 662/1318] 10.222.0.40 () {58 vars in 1061 bytes} [Thu Jun 16 11:47:52 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:47:52.337729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22baeb-c8de-46da-a172-473fe85095c1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:47:52.340056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7e22baeb-c8de-46da-a172-473fe85095c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:52.349732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:52.439571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e22baeb-c8de-46da-a172-473fe85095c1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:47:52.519612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:47:52.520289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:52.520577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:52.520857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:52.522492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:52.584588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:52.630718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:47:52.632197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efe5b94-f8e4-4d77-9dd3-27706644ad2d service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:47:52.650209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 657/1319] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:47:52 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 313 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:47:55.849638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cb93-f4ea-407e-8d9b-e4392c32a9f1 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:55.850151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cb93-f4ea-407e-8d9b-e4392c32a9f1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:55.850151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91b9cb93-f4ea-407e-8d9b-e4392c32a9f1 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:55.851502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 663/1320] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:47:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:55.856565 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243cdeb-99c3-4974-bb9c-6be1dcc41711 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:55.856998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243cdeb-99c3-4974-bb9c-6be1dcc41711 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:55.857136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4243cdeb-99c3-4974-bb9c-6be1dcc41711 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:55.924787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4243cdeb-99c3-4974-bb9c-6be1dcc41711 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:55.925998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4243cdeb-99c3-4974-bb9c-6be1dcc41711 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:56.019479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 658/1321] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:47:55 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:47:56.058395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cb93-f4ea-407e-8d9b-e4392c32a9f1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:47:56.059958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-91b9cb93-f4ea-407e-8d9b-e4392c32a9f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:56.108919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:56.160546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91b9cb93-f4ea-407e-8d9b-e4392c32a9f1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:47:56.233801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:47:56.234446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:56.234756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:56.235053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:56.236728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:56.331407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:56.345191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:47:56.346626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ffd5bc-6270-4c45-a683-1387bba6b697 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:47:56.365422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 664/1322] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:47:56 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:47:59.524342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b05d-8f47-4fb8-9327-3eaac3a7d1df None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:59.524873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b05d-8f47-4fb8-9327-3eaac3a7d1df None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:59.524873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f95b05d-8f47-4fb8-9327-3eaac3a7d1df None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:59.526955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 659/1323] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:47:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:47:59.532140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755320f3-1c07-4ff6-bfd5-abf48f5f58c4 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:59.532557 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755320f3-1c07-4ff6-bfd5-abf48f5f58c4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:59.532725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755320f3-1c07-4ff6-bfd5-abf48f5f58c4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:59.605894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-755320f3-1c07-4ff6-bfd5-abf48f5f58c4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:47:59.607272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-755320f3-1c07-4ff6-bfd5-abf48f5f58c4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:59.717015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 665/1324] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:47:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:47:59.753225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b05d-8f47-4fb8-9327-3eaac3a7d1df None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:47:59.754814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9f95b05d-8f47-4fb8-9327-3eaac3a7d1df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:47:59.776634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:47:59.861556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f95b05d-8f47-4fb8-9327-3eaac3a7d1df None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:47:59.934267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:47:59.934922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:47:59.935208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:47:59.935483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:47:59.937197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:00.011338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:00.050046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:00.051490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef93f76b-a6ea-4adc-a22d-588b46c2dcbb service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:00.069694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 660/1325] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:47:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 317 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:02.254808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de29a0-27ff-493e-83d1-3cbac7ae88c1 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:02.255348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de29a0-27ff-493e-83d1-3cbac7ae88c1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:02.255348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de29a0-27ff-493e-83d1-3cbac7ae88c1 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:02.256774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 666/1326] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:02.261901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833955-5504-49e4-93f3-4f2d1f5c8f5d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:02.262326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833955-5504-49e4-93f3-4f2d1f5c8f5d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:02.262541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77833955-5504-49e4-93f3-4f2d1f5c8f5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:02.327884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-77833955-5504-49e4-93f3-4f2d1f5c8f5d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:02.329107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-77833955-5504-49e4-93f3-4f2d1f5c8f5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:02.428263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 661/1327] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:02.463851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de29a0-27ff-493e-83d1-3cbac7ae88c1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:02.465504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e3de29a0-27ff-493e-83d1-3cbac7ae88c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:02.532973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:02.576645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de29a0-27ff-493e-83d1-3cbac7ae88c1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:02.652168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:02.652838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:02.653127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:02.653460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:02.655257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:02.775174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:02.778330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:02.779775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857034cf-a886-4ce1-9689-d9f12094b28b service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:02.797725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 667/1328] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:04.963091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ef28f-beb6-4812-bb41-a3af1084a508 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:04.963618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ef28f-beb6-4812-bb41-a3af1084a508 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:04.963618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6ef28f-beb6-4812-bb41-a3af1084a508 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:04.965056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 662/1329] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:04.970604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f34af-0568-46a9-9500-5be9c141f49e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:04.971011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f34af-0568-46a9-9500-5be9c141f49e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:04.971170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb3f34af-0568-46a9-9500-5be9c141f49e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:05.035801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-bb3f34af-0568-46a9-9500-5be9c141f49e None None] MFA Rules not processed for user `05e24c3335b843adafa09ce1a2c81793`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:05.037000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bb3f34af-0568-46a9-9500-5be9c141f49e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:05.133184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 668/1330] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:04 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 164 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:05.169897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ef28f-beb6-4812-bb41-a3af1084a508 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:05.171409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5f6ef28f-beb6-4812-bb41-a3af1084a508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:05.221379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:05.297589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6ef28f-beb6-4812-bb41-a3af1084a508 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:05.373874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:05.374529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:05.374811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:05.375083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:05.376751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:05.452336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:05.484862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:05.486335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91ad68dd-0d00-4cc7-a1f4-41c8f44e7b1e service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:05.508249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 663/1331] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:05 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:08.065327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d95c3-ab96-4617-82be-8766eb5db036 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:08.065887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d95c3-ab96-4617-82be-8766eb5db036 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:08.065887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-740d95c3-ab96-4617-82be-8766eb5db036 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:08.067261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 669/1332] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:48:08.073640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d7290-b6b4-4e3f-9fff-364bfde639b8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:08.073640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d7290-b6b4-4e3f-9fff-364bfde639b8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:08.073640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b2d7290-b6b4-4e3f-9fff-364bfde639b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:08.137579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7b2d7290-b6b4-4e3f-9fff-364bfde639b8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:08.138781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7b2d7290-b6b4-4e3f-9fff-364bfde639b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:08.233300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 664/1333] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:08.268494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d95c3-ab96-4617-82be-8766eb5db036 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:08.270036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-740d95c3-ab96-4617-82be-8766eb5db036 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:08.339953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:08.373877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-740d95c3-ab96-4617-82be-8766eb5db036 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:08.445575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:08.446297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:08.446612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:08.446933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:08.448758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:08.555711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:08.557217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37db9bcd-627a-451a-94be-b0c0e47052ef service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:08.567022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:08.574953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 670/1334] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:10.715821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a48b39-fd43-4294-99ce-7951d386adb2 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:10.716265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a48b39-fd43-4294-99ce-7951d386adb2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:10.716405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39a48b39-fd43-4294-99ce-7951d386adb2 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:10.718028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 665/1335] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:10.723048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9e77-5546-4fb4-9341-bf7d28d96a27 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:10.723500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9e77-5546-4fb4-9341-bf7d28d96a27 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:10.723604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98ec9e77-5546-4fb4-9341-bf7d28d96a27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:10.786337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-98ec9e77-5546-4fb4-9341-bf7d28d96a27 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:10.787512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-98ec9e77-5546-4fb4-9341-bf7d28d96a27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:10.881448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 671/1336] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:48:10.920803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a48b39-fd43-4294-99ce-7951d386adb2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:10.922591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-39a48b39-fd43-4294-99ce-7951d386adb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:10.961220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:11.024242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39a48b39-fd43-4294-99ce-7951d386adb2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:11.095998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:11.096639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:11.096920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:11.097193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:11.098802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:11.179474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:11.204308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:11.205733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-744d8e0b-681d-48ca-b75a-c685af9f93a6 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:11.224441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 666/1337] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 304 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:13.866498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b1850-d9d6-44f9-836a-806ffe581846 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:13.867010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b1850-d9d6-44f9-836a-806ffe581846 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:13.867010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b01b1850-d9d6-44f9-836a-806ffe581846 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:13.868399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 672/1338] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:13.873946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960875-2fd1-4d8a-86da-fc85458bd5af None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:13.874365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960875-2fd1-4d8a-86da-fc85458bd5af None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:13.874501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03960875-2fd1-4d8a-86da-fc85458bd5af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:13.937129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-03960875-2fd1-4d8a-86da-fc85458bd5af None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:13.938329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-03960875-2fd1-4d8a-86da-fc85458bd5af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:14.032984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 667/1339] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:13 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:14.069232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b1850-d9d6-44f9-836a-806ffe581846 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:14.070775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b01b1850-d9d6-44f9-836a-806ffe581846 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:14.146049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:14.174572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b01b1850-d9d6-44f9-836a-806ffe581846 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:14.246785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:14.247429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:14.247713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:14.248021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:14.249668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:14.356782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:14.358200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2485fdfc-fa12-44b5-9074-f4204ecac263 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:14.375604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 673/1340] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:14 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 307 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:18.661522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0171a6-2df9-4eae-bde4-b8c2067831f5 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:18.662111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0171a6-2df9-4eae-bde4-b8c2067831f5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:18.662111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab0171a6-2df9-4eae-bde4-b8c2067831f5 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:18.663494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 668/1341] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:18.669180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1ecd-8d2e-4c4d-8373-08998d8d5b04 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:18.670862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1ecd-8d2e-4c4d-8373-08998d8d5b04 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:18.671243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858b1ecd-8d2e-4c4d-8373-08998d8d5b04 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:18.679077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:18.748957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-858b1ecd-8d2e-4c4d-8373-08998d8d5b04 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:18.758981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-858b1ecd-8d2e-4c4d-8373-08998d8d5b04 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:18.855291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 674/1342] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 187 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:18.895857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0171a6-2df9-4eae-bde4-b8c2067831f5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:18.897991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ab0171a6-2df9-4eae-bde4-b8c2067831f5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:18.946561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:18.991264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab0171a6-2df9-4eae-bde4-b8c2067831f5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:18.999051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a64cf9-0695-43d9-8d2f-43783e2c534c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:18.999716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a64cf9-0695-43d9-8d2f-43783e2c534c service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:19.000037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a64cf9-0695-43d9-8d2f-43783e2c534c service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:19.000318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a64cf9-0695-43d9-8d2f-43783e2c534c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:19.010023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a64cf9-0695-43d9-8d2f-43783e2c534c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:19.011452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a64cf9-0695-43d9-8d2f-43783e2c534c service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:19.020992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 669/1343] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:22.103389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8570c-b77d-461f-8b4e-549cd31459e2 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:22.103946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8570c-b77d-461f-8b4e-549cd31459e2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:22.104063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab8570c-b77d-461f-8b4e-549cd31459e2 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:22.105460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 675/1344] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:22.110735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596960b-f423-4516-9fe8-55e629305fb5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:22.111149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596960b-f423-4516-9fe8-55e629305fb5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:22.111307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a596960b-f423-4516-9fe8-55e629305fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:22.140228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a596960b-f423-4516-9fe8-55e629305fb5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:22.141426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a596960b-f423-4516-9fe8-55e629305fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:22.217764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 670/1345] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:22 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:22.255521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8570c-b77d-461f-8b4e-549cd31459e2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:22.257208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1ab8570c-b77d-461f-8b4e-549cd31459e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:22.270126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab8570c-b77d-461f-8b4e-549cd31459e2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:22.273638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8aec90ed-0f04-44ae-bcef-fecd1334f534 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:22.274283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec90ed-0f04-44ae-bcef-fecd1334f534 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:22.274576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec90ed-0f04-44ae-bcef-fecd1334f534 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:22.274854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aec90ed-0f04-44ae-bcef-fecd1334f534 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:22.284120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec90ed-0f04-44ae-bcef-fecd1334f534 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:22.285542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8aec90ed-0f04-44ae-bcef-fecd1334f534 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:22.295668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 676/1346] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:22 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:26.984207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c957529-e62b-4da0-88ef-9926025fca10 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:26.984679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c957529-e62b-4da0-88ef-9926025fca10 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:26.984772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c957529-e62b-4da0-88ef-9926025fca10 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:26.986264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 671/1347] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:26.991564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fc0a4-8584-4ee0-9a52-8fee9c2f2d09 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:26.992039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fc0a4-8584-4ee0-9a52-8fee9c2f2d09 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:26.992272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b77fc0a4-8584-4ee0-9a52-8fee9c2f2d09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:27.016883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b77fc0a4-8584-4ee0-9a52-8fee9c2f2d09 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:27.018110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b77fc0a4-8584-4ee0-9a52-8fee9c2f2d09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:27.027459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 677/1348] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:27.065171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c957529-e62b-4da0-88ef-9926025fca10 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:27.076192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c957529-e62b-4da0-88ef-9926025fca10 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:27.079645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4375994-d9de-4b4f-96f7-7386014052ce service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:27.080309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375994-d9de-4b4f-96f7-7386014052ce service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:27.080593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375994-d9de-4b4f-96f7-7386014052ce service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:27.080869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4375994-d9de-4b4f-96f7-7386014052ce service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:27.090702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375994-d9de-4b4f-96f7-7386014052ce service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:27.092590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4375994-d9de-4b4f-96f7-7386014052ce service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:27.102228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 672/1349] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:32.022868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d4d73-f940-4fed-ad35-50f0232c14c3 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:32.023379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d4d73-f940-4fed-ad35-50f0232c14c3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:32.023496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b29d4d73-f940-4fed-ad35-50f0232c14c3 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:32.025757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 678/1350] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:48:32 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:32.031349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4ed7-9a92-4fda-9f3f-2204f9d71e6a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:32.031799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4ed7-9a92-4fda-9f3f-2204f9d71e6a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:32.031941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99fb4ed7-9a92-4fda-9f3f-2204f9d71e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:32.116845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-99fb4ed7-9a92-4fda-9f3f-2204f9d71e6a None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:32.118324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-99fb4ed7-9a92-4fda-9f3f-2204f9d71e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:32.154552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:32.207603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 673/1351] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:48:32 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:36.308505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403807d-7f76-480e-87f0-a6bca283aa83 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:36.309055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403807d-7f76-480e-87f0-a6bca283aa83 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:36.309055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5403807d-7f76-480e-87f0-a6bca283aa83 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:36.310442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 679/1352] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:48:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:36.315450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00dfd4-1abb-4240-a4d0-7241d89e6eaa None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:36.315894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00dfd4-1abb-4240-a4d0-7241d89e6eaa None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:36.316226 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00dfd4-1abb-4240-a4d0-7241d89e6eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:36.334892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-eb00dfd4-1abb-4240-a4d0-7241d89e6eaa None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:36.336269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-eb00dfd4-1abb-4240-a4d0-7241d89e6eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:36.345632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 674/1353] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:48:36 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:37.010320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=72 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:38.002374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e400b-9890-40d9-b338-bd909696c569 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:38.003019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e400b-9890-40d9-b338-bd909696c569 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:38.003019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981e400b-9890-40d9-b338-bd909696c569 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:38.004358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 680/1354] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:48:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:38.009413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353e9a-0b65-4c02-bd8a-ca55de018497 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:38.009841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353e9a-0b65-4c02-bd8a-ca55de018497 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:38.009955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe353e9a-0b65-4c02-bd8a-ca55de018497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:38.027964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-fe353e9a-0b65-4c02-bd8a-ca55de018497 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:38.029161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fe353e9a-0b65-4c02-bd8a-ca55de018497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:38.037762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 675/1355] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:48:38 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:48:41.716334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c87f3a-faf7-4596-b042-541025d4312c None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:41.717143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c87f3a-faf7-4596-b042-541025d4312c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:41.717143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61c87f3a-faf7-4596-b042-541025d4312c None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:41.718551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 681/1356] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:41.723857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd5617-53c4-4485-94fa-f65c9e8a50d5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:41.724302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd5617-53c4-4485-94fa-f65c9e8a50d5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:41.724462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd5617-53c4-4485-94fa-f65c9e8a50d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:41.743852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-64dd5617-53c4-4485-94fa-f65c9e8a50d5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:41.745004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-64dd5617-53c4-4485-94fa-f65c9e8a50d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:41.753280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 676/1357] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:41.808038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a495e16-3c4a-44bd-95c0-ffbf6ed2a5d5 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:41.808565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a495e16-3c4a-44bd-95c0-ffbf6ed2a5d5 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:41.809008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a495e16-3c4a-44bd-95c0-ffbf6ed2a5d5 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:41.810772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 682/1358] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 11:48:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:41.824938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d75e-2974-4f07-840a-a726e09edee5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:41.825504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d75e-2974-4f07-840a-a726e09edee5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:41.825991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b5d75e-2974-4f07-840a-a726e09edee5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:41.883072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-79b5d75e-2974-4f07-840a-a726e09edee5 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:41.884492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-79b5d75e-2974-4f07-840a-a726e09edee5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:41.972482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 677/1359] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 11:48:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 149 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:48:41.980811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a495e16-3c4a-44bd-95c0-ffbf6ed2a5d5 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:41.996052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a495e16-3c4a-44bd-95c0-ffbf6ed2a5d5 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:42.001452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc520061-cc6b-4375-b6fa-f970069d0a55 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:42.002099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520061-cc6b-4375-b6fa-f970069d0a55 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:42.002389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520061-cc6b-4375-b6fa-f970069d0a55 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:42.002671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc520061-cc6b-4375-b6fa-f970069d0a55 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:42.011224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc520061-cc6b-4375-b6fa-f970069d0a55 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:42.012848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc520061-cc6b-4375-b6fa-f970069d0a55 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:42.022726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 683/1360] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:44.357941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01656d0-fb9a-45a9-9a63-acb6ea0a1fcf None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:44.358421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01656d0-fb9a-45a9-9a63-acb6ea0a1fcf None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:44.358500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01656d0-fb9a-45a9-9a63-acb6ea0a1fcf None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:44.359825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 678/1361] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:48:44.365203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67de84d-2efd-4b4d-a335-9e65dc8da508 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:44.365655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67de84d-2efd-4b4d-a335-9e65dc8da508 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:44.365760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67de84d-2efd-4b4d-a335-9e65dc8da508 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:44.385303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d67de84d-2efd-4b4d-a335-9e65dc8da508 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:44.386622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d67de84d-2efd-4b4d-a335-9e65dc8da508 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:44.395544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 684/1362] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:44.442493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01656d0-fb9a-45a9-9a63-acb6ea0a1fcf None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:44.458022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01656d0-fb9a-45a9-9a63-acb6ea0a1fcf None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:44.464788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fe91a24-d451-423c-938d-2700bb261bd9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:44.465436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91a24-d451-423c-938d-2700bb261bd9 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:44.465721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91a24-d451-423c-938d-2700bb261bd9 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:44.466000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fe91a24-d451-423c-938d-2700bb261bd9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:44.476794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe91a24-d451-423c-938d-2700bb261bd9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:44.480011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fe91a24-d451-423c-938d-2700bb261bd9 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:44.491177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 679/1363] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:47.017923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2137d43-0613-4a8f-9cd7-06792b0cf6bc None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:47.018389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2137d43-0613-4a8f-9cd7-06792b0cf6bc None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:47.018389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2137d43-0613-4a8f-9cd7-06792b0cf6bc None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:47.019860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 685/1364] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:47.025144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9371-f2d2-484d-b876-3d05bdf6eff6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:47.025560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9371-f2d2-484d-b876-3d05bdf6eff6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:47.025706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cad9371-f2d2-484d-b876-3d05bdf6eff6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:47.044057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8cad9371-f2d2-484d-b876-3d05bdf6eff6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:47.045338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8cad9371-f2d2-484d-b876-3d05bdf6eff6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:47.055656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 680/1365] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:47.103294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a3b2a-bbcc-43a4-a2d2-4f5ca5cfedfd None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:47.103781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a3b2a-bbcc-43a4-a2d2-4f5ca5cfedfd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:47.104110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135a3b2a-bbcc-43a4-a2d2-4f5ca5cfedfd None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:47.105506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 686/1366] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 11:48:47 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:47.110883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3f5de-2ded-4df2-b046-8c9091de3ad5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:47.111179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3f5de-2ded-4df2-b046-8c9091de3ad5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:47.112324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf3f5de-2ded-4df2-b046-8c9091de3ad5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:47.130088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ddf3f5de-2ded-4df2-b046-8c9091de3ad5 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:47.131469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ddf3f5de-2ded-4df2-b046-8c9091de3ad5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:47.141808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 681/1367] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 11:48:47 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:47.154541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a3b2a-bbcc-43a4-a2d2-4f5ca5cfedfd None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:47.170286 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135a3b2a-bbcc-43a4-a2d2-4f5ca5cfedfd None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:47.184956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4bba1a5-a5a0-4295-b097-851ca2544cda service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:47.186004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bba1a5-a5a0-4295-b097-851ca2544cda service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:47.187397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bba1a5-a5a0-4295-b097-851ca2544cda service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:47.187918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4bba1a5-a5a0-4295-b097-851ca2544cda service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:47.205218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bba1a5-a5a0-4295-b097-851ca2544cda service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:47.207808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4bba1a5-a5a0-4295-b097-851ca2544cda service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:47.222102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 687/1368] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:49.556922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a675d2-506d-4fd2-8406-35d0a1c939c9 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:49.557550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a675d2-506d-4fd2-8406-35d0a1c939c9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:49.557550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21a675d2-506d-4fd2-8406-35d0a1c939c9 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:49.558856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 682/1369] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:49.564179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ba48c-a0ba-4fbe-ba1f-611f0f0416e0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:49.564975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ba48c-a0ba-4fbe-ba1f-611f0f0416e0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:49.565432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a3ba48c-a0ba-4fbe-ba1f-611f0f0416e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:49.583848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-0a3ba48c-a0ba-4fbe-ba1f-611f0f0416e0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:49.585344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0a3ba48c-a0ba-4fbe-ba1f-611f0f0416e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:49.594451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 688/1370] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:48:49.638461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a675d2-506d-4fd2-8406-35d0a1c939c9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:49.653029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21a675d2-506d-4fd2-8406-35d0a1c939c9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:49.658369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9590f6-9812-4b6e-9b1e-a1dbd2e8c69d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:49.659007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9590f6-9812-4b6e-9b1e-a1dbd2e8c69d service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:49.659289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9590f6-9812-4b6e-9b1e-a1dbd2e8c69d service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:49.659650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9590f6-9812-4b6e-9b1e-a1dbd2e8c69d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:49.668111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9590f6-9812-4b6e-9b1e-a1dbd2e8c69d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:49.669820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9590f6-9812-4b6e-9b1e-a1dbd2e8c69d service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:49.682288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 683/1371] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:49.885250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f93f7f-8ee1-4f65-b547-ae2f376b6ed7 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:49.885543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f93f7f-8ee1-4f65-b547-ae2f376b6ed7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:49.886018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f93f7f-8ee1-4f65-b547-ae2f376b6ed7 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:49.887656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 689/1372] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:48:49.893516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65737a8-934e-4989-836b-63a7e5a08b97 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:49.894017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65737a8-934e-4989-836b-63a7e5a08b97 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:49.894467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f65737a8-934e-4989-836b-63a7e5a08b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:49.913378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f65737a8-934e-4989-836b-63a7e5a08b97 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:49.914992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f65737a8-934e-4989-836b-63a7e5a08b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:49.923370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 684/1373] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:50.230664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa8d08-ea7d-4188-ae4c-6f804d869f34 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:50.230937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa8d08-ea7d-4188-ae4c-6f804d869f34 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:50.231220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbfa8d08-ea7d-4188-ae4c-6f804d869f34 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:50.232782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 690/1374] 10.222.0.40 () {54 vars in 963 bytes} [Thu Jun 16 11:48:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:50.237596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ba8fd-0fb3-491b-ad14-67d0a402b439 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:50.238534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ba8fd-0fb3-491b-ad14-67d0a402b439 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:50.238534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd4ba8fd-0fb3-491b-ad14-67d0a402b439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:50.245733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:48:50.288927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-fd4ba8fd-0fb3-491b-ad14-67d0a402b439 None None] MFA Rules not processed for user `72c24cb67566467dbdf65cfd44a59274`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:50.290730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fd4ba8fd-0fb3-491b-ad14-67d0a402b439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:50.356023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 685/1375] 10.222.0.40 () {58 vars in 1060 bytes} [Thu Jun 16 11:48:50 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:50.361209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa8d08-ea7d-4188-ae4c-6f804d869f34 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:50.372962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbfa8d08-ea7d-4188-ae4c-6f804d869f34 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:50.376657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cc68691-bb9c-41a5-aac7-0dd9c0f84403 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:50.377411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc68691-bb9c-41a5-aac7-0dd9c0f84403 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:50.377812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc68691-bb9c-41a5-aac7-0dd9c0f84403 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:50.378195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cc68691-bb9c-41a5-aac7-0dd9c0f84403 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:50.388104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc68691-bb9c-41a5-aac7-0dd9c0f84403 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:50.389872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cc68691-bb9c-41a5-aac7-0dd9c0f84403 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:50.400598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 691/1376] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:48:51.951853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c8277-fc98-4330-94da-dea07379ce03 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:51.952400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c8277-fc98-4330-94da-dea07379ce03 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:51.952514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3c8277-fc98-4330-94da-dea07379ce03 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:51.953817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 686/1377] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:51.958962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ebfe-b2c0-4983-a385-52fd606520a0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:51.959356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ebfe-b2c0-4983-a385-52fd606520a0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:51.959522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b538ebfe-b2c0-4983-a385-52fd606520a0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:51.979184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b538ebfe-b2c0-4983-a385-52fd606520a0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:51.980406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b538ebfe-b2c0-4983-a385-52fd606520a0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:51.988186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 692/1378] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:52.031534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c8277-fc98-4330-94da-dea07379ce03 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:52.046292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3c8277-fc98-4330-94da-dea07379ce03 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:52.051940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93ebe90-c6e8-4c9d-9c10-8b321ec04691 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:52.052576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ebe90-c6e8-4c9d-9c10-8b321ec04691 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:52.052861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ebe90-c6e8-4c9d-9c10-8b321ec04691 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:52.053139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93ebe90-c6e8-4c9d-9c10-8b321ec04691 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:52.061119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ebe90-c6e8-4c9d-9c10-8b321ec04691 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:52.062694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93ebe90-c6e8-4c9d-9c10-8b321ec04691 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:52.072387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 687/1379] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:54.367968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efad04-4023-4956-8da6-be2fcf73d177 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:54.368499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efad04-4023-4956-8da6-be2fcf73d177 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:54.368499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66efad04-4023-4956-8da6-be2fcf73d177 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:54.369999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 693/1380] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:54.375277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0ba07-3d06-44ea-acbc-314e686a9bd7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:54.375673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0ba07-3d06-44ea-acbc-314e686a9bd7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:54.375905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d0ba07-3d06-44ea-acbc-314e686a9bd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:54.393715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a0d0ba07-3d06-44ea-acbc-314e686a9bd7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:54.394903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a0d0ba07-3d06-44ea-acbc-314e686a9bd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:54.403433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 688/1381] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:54.447567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efad04-4023-4956-8da6-be2fcf73d177 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:54.462738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66efad04-4023-4956-8da6-be2fcf73d177 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:54.468664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19e91b37-3ef5-4f61-aa96-470b1b152e0f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:54.469461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e91b37-3ef5-4f61-aa96-470b1b152e0f service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:54.469912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e91b37-3ef5-4f61-aa96-470b1b152e0f service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:54.470303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19e91b37-3ef5-4f61-aa96-470b1b152e0f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:54.479995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e91b37-3ef5-4f61-aa96-470b1b152e0f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:54.481624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19e91b37-3ef5-4f61-aa96-470b1b152e0f service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:54.492779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 694/1382] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:56.749173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5823174-decc-483f-9241-5a3d0e12f3af None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:56.749697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5823174-decc-483f-9241-5a3d0e12f3af None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:56.749697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5823174-decc-483f-9241-5a3d0e12f3af None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:56.751129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 689/1383] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:56.756426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0a3e3-fcdf-454b-97cc-aced850692a9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:56.756856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0a3e3-fcdf-454b-97cc-aced850692a9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:56.756994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47b0a3e3-fcdf-454b-97cc-aced850692a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:56.775927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-47b0a3e3-fcdf-454b-97cc-aced850692a9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:56.777120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-47b0a3e3-fcdf-454b-97cc-aced850692a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:56.785021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 695/1384] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:56.826224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5823174-decc-483f-9241-5a3d0e12f3af None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:56.841856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5823174-decc-483f-9241-5a3d0e12f3af None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:56.847731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-613000da-5c21-4986-a04d-51554e833139 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:56.848488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613000da-5c21-4986-a04d-51554e833139 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:56.848838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613000da-5c21-4986-a04d-51554e833139 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:56.849185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-613000da-5c21-4986-a04d-51554e833139 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:56.857835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613000da-5c21-4986-a04d-51554e833139 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:56.859484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-613000da-5c21-4986-a04d-51554e833139 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:56.869689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 690/1385] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:48:59.196150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428778-4741-4590-835f-401cbce50657 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:59.196740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428778-4741-4590-835f-401cbce50657 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:59.196740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de428778-4741-4590-835f-401cbce50657 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:59.198135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 696/1386] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:48:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:48:59.203437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd3c68-f807-48f7-9e41-e0cb08f5e03c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:59.203897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd3c68-f807-48f7-9e41-e0cb08f5e03c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:59.204016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cd3c68-f807-48f7-9e41-e0cb08f5e03c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:59.222666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b9cd3c68-f807-48f7-9e41-e0cb08f5e03c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:48:59.223895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b9cd3c68-f807-48f7-9e41-e0cb08f5e03c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:48:59.232633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 691/1387] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:48:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:48:59.276448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428778-4741-4590-835f-401cbce50657 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:48:59.291378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de428778-4741-4590-835f-401cbce50657 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:48:59.296831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9008d9ac-af93-4695-9049-6bda056fb55f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:48:59.297470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008d9ac-af93-4695-9049-6bda056fb55f service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:48:59.297793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008d9ac-af93-4695-9049-6bda056fb55f service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:48:59.298033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9008d9ac-af93-4695-9049-6bda056fb55f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:48:59.306261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008d9ac-af93-4695-9049-6bda056fb55f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:48:59.307829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9008d9ac-af93-4695-9049-6bda056fb55f service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:48:59.317190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 697/1388] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:48:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:02.183438 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d5725-b450-4bf0-806c-898417bc78f4 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:02.184021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d5725-b450-4bf0-806c-898417bc78f4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:02.184021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-063d5725-b450-4bf0-806c-898417bc78f4 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:02.185409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 692/1389] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:49:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:02.191212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd81b9-521b-423a-a99d-5da2e0783509 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:02.191647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd81b9-521b-423a-a99d-5da2e0783509 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:02.191965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dd81b9-521b-423a-a99d-5da2e0783509 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:02.213954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b7dd81b9-521b-423a-a99d-5da2e0783509 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:02.215923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b7dd81b9-521b-423a-a99d-5da2e0783509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:02.226091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 698/1390] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:49:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:02.279527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d5725-b450-4bf0-806c-898417bc78f4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:02.294944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-063d5725-b450-4bf0-806c-898417bc78f4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:02.300494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cbde985-ff13-49a6-b35c-e40360913067 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:02.301138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde985-ff13-49a6-b35c-e40360913067 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:02.301436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde985-ff13-49a6-b35c-e40360913067 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:02.301766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cbde985-ff13-49a6-b35c-e40360913067 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:02.310780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbde985-ff13-49a6-b35c-e40360913067 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:02.312406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cbde985-ff13-49a6-b35c-e40360913067 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:02.321962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 693/1391] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:04.608632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2333-e920-4398-904e-af3881a297a2 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:04.609558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2333-e920-4398-904e-af3881a297a2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:04.609558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9b2333-e920-4398-904e-af3881a297a2 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:04.611006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 699/1392] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:49:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:04.616431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d95a2-31dc-44ef-9b47-ae08c296bff0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:04.617350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d95a2-31dc-44ef-9b47-ae08c296bff0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:04.617520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068d95a2-31dc-44ef-9b47-ae08c296bff0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:04.635704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-068d95a2-31dc-44ef-9b47-ae08c296bff0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:04.636914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-068d95a2-31dc-44ef-9b47-ae08c296bff0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:04.644579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 694/1393] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:49:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:04.687823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2333-e920-4398-904e-af3881a297a2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:04.703164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9b2333-e920-4398-904e-af3881a297a2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:04.708604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0d3614f-7f09-48bc-b5e5-1be891070b5b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:04.709253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3614f-7f09-48bc-b5e5-1be891070b5b service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:04.709542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3614f-7f09-48bc-b5e5-1be891070b5b service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:04.709823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0d3614f-7f09-48bc-b5e5-1be891070b5b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:04.718234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3614f-7f09-48bc-b5e5-1be891070b5b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:04.719825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0d3614f-7f09-48bc-b5e5-1be891070b5b service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:04.729282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 700/1394] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:05.934813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a0c5-7b9c-4c1f-9150-40ade6f34c82 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:05.935336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a0c5-7b9c-4c1f-9150-40ade6f34c82 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:05.935336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e58a0c5-7b9c-4c1f-9150-40ade6f34c82 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:05.936825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 695/1395] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:49:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:05.942105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f808dd9-be37-4cb1-8ee7-115d670f7ddf None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:05.942540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f808dd9-be37-4cb1-8ee7-115d670f7ddf None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:05.942666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f808dd9-be37-4cb1-8ee7-115d670f7ddf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:05.961692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1f808dd9-be37-4cb1-8ee7-115d670f7ddf None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:05.962865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1f808dd9-be37-4cb1-8ee7-115d670f7ddf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:05.971085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 701/1396] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:49:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:06.028929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a2ad0-b7d7-4e68-a0d9-afd726c5fb2d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:06.029224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a2ad0-b7d7-4e68-a0d9-afd726c5fb2d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:06.029516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-176a2ad0-b7d7-4e68-a0d9-afd726c5fb2d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:06.030861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 696/1397] 10.222.0.40 () {54 vars in 962 bytes} [Thu Jun 16 11:49:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:06.035965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffbe4d-0155-4d9f-a6dd-d07dcf5c28d6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:06.036188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffbe4d-0155-4d9f-a6dd-d07dcf5c28d6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:06.036467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffffbe4d-0155-4d9f-a6dd-d07dcf5c28d6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:06.083248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ffffbe4d-0155-4d9f-a6dd-d07dcf5c28d6 None None] MFA Rules not processed for user `35d68b3bee5445849c59da135df06fea`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:06.084457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ffffbe4d-0155-4d9f-a6dd-d07dcf5c28d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:06.146331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 702/1398] 10.222.0.40 () {58 vars in 1059 bytes} [Thu Jun 16 11:49:06 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:06.153933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a2ad0-b7d7-4e68-a0d9-afd726c5fb2d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:06.165100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-176a2ad0-b7d7-4e68-a0d9-afd726c5fb2d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:06.168807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27d9b3e3-e363-439b-9b3a-b9e8d1792bc2 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:06.169454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9b3e3-e363-439b-9b3a-b9e8d1792bc2 service glance] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:06.170001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9b3e3-e363-439b-9b3a-b9e8d1792bc2 service glance] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:06.170346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27d9b3e3-e363-439b-9b3a-b9e8d1792bc2 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:06.182090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9b3e3-e363-439b-9b3a-b9e8d1792bc2 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:06.183998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27d9b3e3-e363-439b-9b3a-b9e8d1792bc2 service glance] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:06.196236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 697/1399] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:07.040448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9b273-f3fe-4613-86b3-fff709535485 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.040891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9b273-f3fe-4613-86b3-fff709535485 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.041258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ac9b273-f3fe-4613-86b3-fff709535485 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.042548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebba50-d59d-458f-95e8-ba647de7baa6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.042931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 703/1400] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:49:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:07.043026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebba50-d59d-458f-95e8-ba647de7baa6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.043168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ebba50-d59d-458f-95e8-ba647de7baa6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.047461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.047757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.048078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.054369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:49:07.067006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:07.068265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:07.076632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 704/1401] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:49:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:49:07.092736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-17ebba50-d59d-458f-95e8-ba647de7baa6 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:07.093960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-17ebba50-d59d-458f-95e8-ba647de7baa6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:07.121625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:07.133814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd64b862-8815-42ff-8962-9fcd23eca68a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:07.134412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:49:07.139471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b726228-3db7-4d7d-b974-b30c713f7b4a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:07.140148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b726228-3db7-4d7d-b974-b30c713f7b4a service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.140439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b726228-3db7-4d7d-b974-b30c713f7b4a service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.140722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b726228-3db7-4d7d-b974-b30c713f7b4a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.149544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b726228-3db7-4d7d-b974-b30c713f7b4a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:07.151061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b726228-3db7-4d7d-b974-b30c713f7b4a service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:07.158134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 698/1402] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:49:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:07.160821 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 705/1403] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:49:07.198688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f533d65-e83c-4acf-b246-f5b9a9640c1a None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.198981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f533d65-e83c-4acf-b246-f5b9a9640c1a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.199264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f533d65-e83c-4acf-b246-f5b9a9640c1a None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.200684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 699/1404] 10.222.0.40 () {54 vars in 942 bytes} [Thu Jun 16 11:49:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:07.207688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec886a-b49e-491f-be3e-040ed612cdaa None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.208018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec886a-b49e-491f-be3e-040ed612cdaa None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.208299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bec886a-b49e-491f-be3e-040ed612cdaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.256953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8bec886a-b49e-491f-be3e-040ed612cdaa None None] MFA Rules not processed for user `7740886489ae4fe78a1d81cd12852750`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:07.258702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8bec886a-b49e-491f-be3e-040ed612cdaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:07.329661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 706/1405] 10.222.0.40 () {58 vars in 1039 bytes} [Thu Jun 16 11:49:07 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:49:07.339589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f533d65-e83c-4acf-b246-f5b9a9640c1a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:07.352987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f533d65-e83c-4acf-b246-f5b9a9640c1a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:07.357865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71dfce-5516-498b-aed7-48b6f6407839 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:07.358535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71dfce-5516-498b-aed7-48b6f6407839 service swift] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:07.358817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71dfce-5516-498b-aed7-48b6f6407839 service swift] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:07.359103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71dfce-5516-498b-aed7-48b6f6407839 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:07.367247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71dfce-5516-498b-aed7-48b6f6407839 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:07.368993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71dfce-5516-498b-aed7-48b6f6407839 service swift] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:07.380000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 700/1406] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:49:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:49:09.458069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e069c-f95c-4a41-852e-03c50b793692 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:09.458623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e069c-f95c-4a41-852e-03c50b793692 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:09.458623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b37e069c-f95c-4a41-852e-03c50b793692 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:09.464840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 707/1407] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:49:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:09.468833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002344ae-4e5f-4bc5-9fd3-799359b3e896 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:09.469300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002344ae-4e5f-4bc5-9fd3-799359b3e896 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:09.469480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-002344ae-4e5f-4bc5-9fd3-799359b3e896 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:09.496337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-002344ae-4e5f-4bc5-9fd3-799359b3e896 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:09.497660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-002344ae-4e5f-4bc5-9fd3-799359b3e896 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:09.507906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 701/1408] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:49:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:09.557756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e069c-f95c-4a41-852e-03c50b793692 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:09.574937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b37e069c-f95c-4a41-852e-03c50b793692 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:09.579937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-290c83cd-ce77-4a1b-98f8-5aa4148fe75f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:09.580600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c83cd-ce77-4a1b-98f8-5aa4148fe75f service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:09.580885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c83cd-ce77-4a1b-98f8-5aa4148fe75f service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:09.581163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290c83cd-ce77-4a1b-98f8-5aa4148fe75f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:09.590572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c83cd-ce77-4a1b-98f8-5aa4148fe75f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:09.592497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-290c83cd-ce77-4a1b-98f8-5aa4148fe75f service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:09.602398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 708/1409] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:19.490429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=18 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:49:19.598357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=58 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:49:31.455217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6e7e-2e1d-41fc-a451-70dc834f4190 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:31.455853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6e7e-2e1d-41fc-a451-70dc834f4190 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:31.455853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3eb6e7e-2e1d-41fc-a451-70dc834f4190 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:31.457201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 702/1410] 10.222.0.40 () {54 vars in 918 bytes} [Thu Jun 16 11:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:31.462587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67c90f-6aa8-4231-b62b-89ec9975ee35 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:31.463045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67c90f-6aa8-4231-b62b-89ec9975ee35 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:31.463153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac67c90f-6aa8-4231-b62b-89ec9975ee35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:31.481469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ac67c90f-6aa8-4231-b62b-89ec9975ee35 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:31.482651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ac67c90f-6aa8-4231-b62b-89ec9975ee35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:31.490969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 709/1411] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jun 16 11:49:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:31.504444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34d26-c11e-4570-b093-652f1122abeb None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:31.504731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34d26-c11e-4570-b093-652f1122abeb None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:31.505002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a34d26-c11e-4570-b093-652f1122abeb None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:31.506323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 703/1412] 10.222.0.40 () {54 vars in 918 bytes} [Thu Jun 16 11:49:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:31.510691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ce019-acbb-4cb3-abc9-ff9d25e3d5db None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:31.511003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ce019-acbb-4cb3-abc9-ff9d25e3d5db None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:31.511280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812ce019-acbb-4cb3-abc9-ff9d25e3d5db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:31.529011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-812ce019-acbb-4cb3-abc9-ff9d25e3d5db None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:31.530215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-812ce019-acbb-4cb3-abc9-ff9d25e3d5db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:31.539714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 710/1413] 10.222.0.40 () {58 vars in 1015 bytes} [Thu Jun 16 11:49:31 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:49:34.129234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42c4f4-d23c-4c46-b939-81bb7541d3f6 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:34.129776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42c4f4-d23c-4c46-b939-81bb7541d3f6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:34.129776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e42c4f4-d23c-4c46-b939-81bb7541d3f6 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:34.131157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 704/1414] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:34.136462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaa861-271e-4c22-936f-a1880bd8fce4 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:34.136841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaa861-271e-4c22-936f-a1880bd8fce4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:34.137028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8aaaa861-271e-4c22-936f-a1880bd8fce4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:34.154334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8aaaa861-271e-4c22-936f-a1880bd8fce4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:34.155526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8aaaa861-271e-4c22-936f-a1880bd8fce4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:34.163834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 711/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:34.205915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42c4f4-d23c-4c46-b939-81bb7541d3f6 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:34.220397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e42c4f4-d23c-4c46-b939-81bb7541d3f6 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:34.226398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f38fb705-e7b9-4a94-a7a8-6bbb08dd77a5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:34.227028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fb705-e7b9-4a94-a7a8-6bbb08dd77a5 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:34.227314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fb705-e7b9-4a94-a7a8-6bbb08dd77a5 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:34.227594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f38fb705-e7b9-4a94-a7a8-6bbb08dd77a5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:34.235579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fb705-e7b9-4a94-a7a8-6bbb08dd77a5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:34.237182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f38fb705-e7b9-4a94-a7a8-6bbb08dd77a5 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:34.247139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 705/1416] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:34 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:35.181286 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d0864-7df9-4ddc-9357-6ccc47c670fe None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:35.181896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d0864-7df9-4ddc-9357-6ccc47c670fe None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:35.181896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e8d0864-7df9-4ddc-9357-6ccc47c670fe None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:35.183203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 712/1417] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:35.188019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ceb79-43e5-40c0-9a19-0d5937ddea06 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:35.188430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ceb79-43e5-40c0-9a19-0d5937ddea06 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:35.188582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e2ceb79-43e5-40c0-9a19-0d5937ddea06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:35.206100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7e2ceb79-43e5-40c0-9a19-0d5937ddea06 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:35.207322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7e2ceb79-43e5-40c0-9a19-0d5937ddea06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:35.215528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 706/1418] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:35.376099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d0864-7df9-4ddc-9357-6ccc47c670fe None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:35.390625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e8d0864-7df9-4ddc-9357-6ccc47c670fe None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:35.396027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be63ec5c-727f-45ea-a27d-c610a7dc1fc2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:35.397263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63ec5c-727f-45ea-a27d-c610a7dc1fc2 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:35.397557 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63ec5c-727f-45ea-a27d-c610a7dc1fc2 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:35.397835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be63ec5c-727f-45ea-a27d-c610a7dc1fc2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:35.405836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63ec5c-727f-45ea-a27d-c610a7dc1fc2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:35.407409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be63ec5c-727f-45ea-a27d-c610a7dc1fc2 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:35.416795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 713/1419] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:36.347951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7991-f563-4867-a32d-e92174c99d10 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:36.348604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7991-f563-4867-a32d-e92174c99d10 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:36.348604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-779a7991-f563-4867-a32d-e92174c99d10 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:36.350408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 707/1420] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:36 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:36.355222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503969ff-ee0d-47bd-88bf-cc2463840f63 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:36.355671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503969ff-ee0d-47bd-88bf-cc2463840f63 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:36.355792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503969ff-ee0d-47bd-88bf-cc2463840f63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:36.373904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-503969ff-ee0d-47bd-88bf-cc2463840f63 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:36.375093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-503969ff-ee0d-47bd-88bf-cc2463840f63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:36.382812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 714/1421] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:36.534552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7991-f563-4867-a32d-e92174c99d10 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:36.549137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-779a7991-f563-4867-a32d-e92174c99d10 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:36.559741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf6247a-5d63-491b-a4ae-f65319fd0488 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:36.559741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6247a-5d63-491b-a4ae-f65319fd0488 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:36.559741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6247a-5d63-491b-a4ae-f65319fd0488 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:36.559741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf6247a-5d63-491b-a4ae-f65319fd0488 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:36.563564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6247a-5d63-491b-a4ae-f65319fd0488 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:36.565170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf6247a-5d63-491b-a4ae-f65319fd0488 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:36.574459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 708/1422] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:37.473957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee68c2-2ae0-41cf-ae25-d477d8bb5d85 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:37.474537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee68c2-2ae0-41cf-ae25-d477d8bb5d85 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:37.474537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aee68c2-2ae0-41cf-ae25-d477d8bb5d85 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:37.475955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 715/1423] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:37.480681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425edf24-f286-42b1-b37d-b596fbcf75fe None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:37.481111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425edf24-f286-42b1-b37d-b596fbcf75fe None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:37.481243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-425edf24-f286-42b1-b37d-b596fbcf75fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:37.499143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-425edf24-f286-42b1-b37d-b596fbcf75fe None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:37.500362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-425edf24-f286-42b1-b37d-b596fbcf75fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:37.508651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 709/1424] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:37.658333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee68c2-2ae0-41cf-ae25-d477d8bb5d85 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:37.673958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aee68c2-2ae0-41cf-ae25-d477d8bb5d85 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:37.679945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7112ce2-9dba-4890-84c3-5902c7ae96c9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:37.680591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7112ce2-9dba-4890-84c3-5902c7ae96c9 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:37.680871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7112ce2-9dba-4890-84c3-5902c7ae96c9 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:37.681145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7112ce2-9dba-4890-84c3-5902c7ae96c9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:37.689446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7112ce2-9dba-4890-84c3-5902c7ae96c9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:37.691010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7112ce2-9dba-4890-84c3-5902c7ae96c9 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:37.700268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 716/1425] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:38.590184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9052-4c4e-4395-90b1-aae9372782ab None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:38.590730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9052-4c4e-4395-90b1-aae9372782ab None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:38.590730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bec9052-4c4e-4395-90b1-aae9372782ab None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:38.592378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 710/1426] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:38.597191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37075e8c-e393-4008-ab13-fa9006acee07 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:38.597589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37075e8c-e393-4008-ab13-fa9006acee07 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:38.597749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37075e8c-e393-4008-ab13-fa9006acee07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:38.617361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-37075e8c-e393-4008-ab13-fa9006acee07 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:38.618550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-37075e8c-e393-4008-ab13-fa9006acee07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:38.626553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 717/1427] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:49:38.777000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9052-4c4e-4395-90b1-aae9372782ab None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:38.791896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bec9052-4c4e-4395-90b1-aae9372782ab None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:38.797408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b6c079-6118-4ece-a075-eafb715ead88 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:38.798279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6c079-6118-4ece-a075-eafb715ead88 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:38.798561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6c079-6118-4ece-a075-eafb715ead88 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:38.798839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b6c079-6118-4ece-a075-eafb715ead88 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:38.807695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6c079-6118-4ece-a075-eafb715ead88 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:38.809341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b6c079-6118-4ece-a075-eafb715ead88 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:38.818980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 711/1428] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:38 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:39.753874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170adbd-09e7-4058-b7ac-5f562eedfe2d None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:39.754437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170adbd-09e7-4058-b7ac-5f562eedfe2d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:39.754437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2170adbd-09e7-4058-b7ac-5f562eedfe2d None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:39.755980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 718/1429] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:39.761284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dd55a-78ca-4c95-8753-d48c12d5c49f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:39.761712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dd55a-78ca-4c95-8753-d48c12d5c49f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:39.761844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79dd55a-78ca-4c95-8753-d48c12d5c49f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:39.779380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d79dd55a-78ca-4c95-8753-d48c12d5c49f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:39.780609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d79dd55a-78ca-4c95-8753-d48c12d5c49f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:39.788255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 712/1430] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:49:39.938679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170adbd-09e7-4058-b7ac-5f562eedfe2d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:39.952743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2170adbd-09e7-4058-b7ac-5f562eedfe2d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:39.958170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-678d5254-8291-4311-b698-23cf311c8c94 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:39.958817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d5254-8291-4311-b698-23cf311c8c94 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:39.959111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d5254-8291-4311-b698-23cf311c8c94 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:39.959388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-678d5254-8291-4311-b698-23cf311c8c94 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:39.967543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d5254-8291-4311-b698-23cf311c8c94 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:39.969156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-678d5254-8291-4311-b698-23cf311c8c94 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:39.978933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 719/1431] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:40.927044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350e2f8-b223-4d76-bcbf-927260da8bda None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:40.927598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350e2f8-b223-4d76-bcbf-927260da8bda None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:40.927598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e350e2f8-b223-4d76-bcbf-927260da8bda None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:40.929025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 713/1432] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:40.933862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a920189-207a-4887-bc9e-8de70ea329cb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:40.934269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a920189-207a-4887-bc9e-8de70ea329cb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:40.934418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a920189-207a-4887-bc9e-8de70ea329cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:40.952320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4a920189-207a-4887-bc9e-8de70ea329cb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:40.953502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4a920189-207a-4887-bc9e-8de70ea329cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:40.961839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 720/1433] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:49:41.113777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350e2f8-b223-4d76-bcbf-927260da8bda None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:41.128358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e350e2f8-b223-4d76-bcbf-927260da8bda None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:41.134297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14078ca6-280c-4416-95b0-3ac6c8a25a1b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:41.134934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14078ca6-280c-4416-95b0-3ac6c8a25a1b service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:41.135220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14078ca6-280c-4416-95b0-3ac6c8a25a1b service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:41.135499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14078ca6-280c-4416-95b0-3ac6c8a25a1b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:41.143485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14078ca6-280c-4416-95b0-3ac6c8a25a1b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:41.145106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14078ca6-280c-4416-95b0-3ac6c8a25a1b service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:41.154349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 714/1434] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:42.027333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627282a-f355-4d6a-82c0-0319ae99f7fc None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:42.027928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627282a-f355-4d6a-82c0-0319ae99f7fc None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:42.027928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b627282a-f355-4d6a-82c0-0319ae99f7fc None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:42.029301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 721/1435] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:42 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:42.034528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4acac-dbff-44ab-a1b2-a0ed5ee88462 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:42.034528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4acac-dbff-44ab-a1b2-a0ed5ee88462 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:42.034980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4c4acac-dbff-44ab-a1b2-a0ed5ee88462 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:42.053002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d4c4acac-dbff-44ab-a1b2-a0ed5ee88462 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:42.054200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d4c4acac-dbff-44ab-a1b2-a0ed5ee88462 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:42.061940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 715/1436] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:42.214579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627282a-f355-4d6a-82c0-0319ae99f7fc None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:42.228886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b627282a-f355-4d6a-82c0-0319ae99f7fc None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:42.234234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a43553f-0488-4e2c-a7f1-957bef40e460 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:42.234872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43553f-0488-4e2c-a7f1-957bef40e460 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:42.235160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43553f-0488-4e2c-a7f1-957bef40e460 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:42.235442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a43553f-0488-4e2c-a7f1-957bef40e460 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:42.243540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43553f-0488-4e2c-a7f1-957bef40e460 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:42.245177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a43553f-0488-4e2c-a7f1-957bef40e460 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:42.254662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 722/1437] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:43.197680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a86a4-c8bc-42d1-b112-bb309284efaf None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:43.198222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a86a4-c8bc-42d1-b112-bb309284efaf None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:43.198222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc5a86a4-c8bc-42d1-b112-bb309284efaf None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:43.199607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 716/1438] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:49:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:43.204633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b7f41-7c7b-4bed-becd-7ea094338adb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:43.205039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b7f41-7c7b-4bed-becd-7ea094338adb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:43.205195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d2b7f41-7c7b-4bed-becd-7ea094338adb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:43.222765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3d2b7f41-7c7b-4bed-becd-7ea094338adb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:43.223985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3d2b7f41-7c7b-4bed-becd-7ea094338adb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:43.232510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 723/1439] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:49:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:43.382414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a86a4-c8bc-42d1-b112-bb309284efaf None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:43.397075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc5a86a4-c8bc-42d1-b112-bb309284efaf None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:43.402604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a782bda3-d735-4f3b-af4b-235c1f8b5a2b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:43.403245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782bda3-d735-4f3b-af4b-235c1f8b5a2b service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:43.403557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782bda3-d735-4f3b-af4b-235c1f8b5a2b service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:43.403839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a782bda3-d735-4f3b-af4b-235c1f8b5a2b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:43.412649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782bda3-d735-4f3b-af4b-235c1f8b5a2b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:43.414233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a782bda3-d735-4f3b-af4b-235c1f8b5a2b service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:43.423779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 717/1440] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:45.428099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a284-db82-45a5-a8eb-bdb23135882a None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:45.428639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a284-db82-45a5-a8eb-bdb23135882a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:45.428639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4316a284-db82-45a5-a8eb-bdb23135882a None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:45.430123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 724/1441] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:45.435275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f568f-f89f-4def-8ecb-5d0aa5cf98b8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:45.435684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f568f-f89f-4def-8ecb-5d0aa5cf98b8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:45.435841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb3f568f-f89f-4def-8ecb-5d0aa5cf98b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:45.453682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-eb3f568f-f89f-4def-8ecb-5d0aa5cf98b8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:45.454930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-eb3f568f-f89f-4def-8ecb-5d0aa5cf98b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:45.463291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 718/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:45.501289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316a284-db82-45a5-a8eb-bdb23135882a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:45.512244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4316a284-db82-45a5-a8eb-bdb23135882a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:45.515722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb0359c-0078-444e-bcc2-d329457146eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:45.516382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0359c-0078-444e-bcc2-d329457146eb service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:45.516666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0359c-0078-444e-bcc2-d329457146eb service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:45.516941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb0359c-0078-444e-bcc2-d329457146eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:45.525705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0359c-0078-444e-bcc2-d329457146eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:45.527116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bb0359c-0078-444e-bcc2-d329457146eb service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:45.536289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 725/1443] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:47.803332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4caf0-b683-41f6-bf1e-6952c54fab32 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:47.803805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4caf0-b683-41f6-bf1e-6952c54fab32 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:47.803982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ae4caf0-b683-41f6-bf1e-6952c54fab32 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:47.805373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 719/1444] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:47.810694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f105619-87e2-4e69-8853-d48c6ded0966 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:47.811119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f105619-87e2-4e69-8853-d48c6ded0966 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:47.811252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f105619-87e2-4e69-8853-d48c6ded0966 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:47.831059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7f105619-87e2-4e69-8853-d48c6ded0966 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:47.832368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7f105619-87e2-4e69-8853-d48c6ded0966 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:47.841417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 726/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:47.879803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4caf0-b683-41f6-bf1e-6952c54fab32 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:47.891113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ae4caf0-b683-41f6-bf1e-6952c54fab32 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:47.894637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb02f17b-27f2-4bca-a18e-bdfd85a523e1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:47.895278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02f17b-27f2-4bca-a18e-bdfd85a523e1 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:47.895566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02f17b-27f2-4bca-a18e-bdfd85a523e1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:47.895973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb02f17b-27f2-4bca-a18e-bdfd85a523e1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:47.905463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02f17b-27f2-4bca-a18e-bdfd85a523e1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:47.907150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb02f17b-27f2-4bca-a18e-bdfd85a523e1 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:47.917518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 720/1446] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:47 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:50.376117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779d507-90db-493a-91a2-8ea53b3ca37f None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:50.376683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779d507-90db-493a-91a2-8ea53b3ca37f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:50.376683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0779d507-90db-493a-91a2-8ea53b3ca37f None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:50.378075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 727/1447] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:49:50.383276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf0dc-b344-417d-9a9d-761d797f5532 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:50.383716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf0dc-b344-417d-9a9d-761d797f5532 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:50.383863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-755bf0dc-b344-417d-9a9d-761d797f5532 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:50.401057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-755bf0dc-b344-417d-9a9d-761d797f5532 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:50.402241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-755bf0dc-b344-417d-9a9d-761d797f5532 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:50.409777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 721/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:50.448228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779d507-90db-493a-91a2-8ea53b3ca37f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:50.458855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0779d507-90db-493a-91a2-8ea53b3ca37f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:50.462409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57748a1b-d1ef-462f-9621-a0ec416ddf3f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:50.463045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748a1b-d1ef-462f-9621-a0ec416ddf3f service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:50.463355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748a1b-d1ef-462f-9621-a0ec416ddf3f service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:50.463628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57748a1b-d1ef-462f-9621-a0ec416ddf3f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:50.472752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748a1b-d1ef-462f-9621-a0ec416ddf3f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:50.474728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57748a1b-d1ef-462f-9621-a0ec416ddf3f service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:50.484307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 728/1449] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:49:52.732491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109c63-8b18-4e2e-9fa9-2a25a343880f None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:52.732997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109c63-8b18-4e2e-9fa9-2a25a343880f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:52.733092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6109c63-8b18-4e2e-9fa9-2a25a343880f None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:52.734403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 722/1450] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:52.739560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1abffe-a980-42e3-9b40-2c19d67721ec None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:52.740017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1abffe-a980-42e3-9b40-2c19d67721ec None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:52.740137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1abffe-a980-42e3-9b40-2c19d67721ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:52.757844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-cc1abffe-a980-42e3-9b40-2c19d67721ec None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:52.759031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cc1abffe-a980-42e3-9b40-2c19d67721ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:52.766765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 729/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:52.804558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109c63-8b18-4e2e-9fa9-2a25a343880f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:52.815112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6109c63-8b18-4e2e-9fa9-2a25a343880f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:52.819162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9a8bb49-801c-40b6-8447-e7e4aae34537 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:52.819797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8bb49-801c-40b6-8447-e7e4aae34537 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:52.820114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8bb49-801c-40b6-8447-e7e4aae34537 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:52.820394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9a8bb49-801c-40b6-8447-e7e4aae34537 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:52.829262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8bb49-801c-40b6-8447-e7e4aae34537 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:52.830746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9a8bb49-801c-40b6-8447-e7e4aae34537 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:52.840753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 723/1452] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:55.013366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe478c-a053-4dc1-ab1e-63d27eca43ec None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:55.013938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe478c-a053-4dc1-ab1e-63d27eca43ec None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:55.013938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabe478c-a053-4dc1-ab1e-63d27eca43ec None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:55.015324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 730/1453] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:55.020503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a666a96-5b71-4342-87c6-3d00a3e4f920 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:55.021527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a666a96-5b71-4342-87c6-3d00a3e4f920 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:55.021634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a666a96-5b71-4342-87c6-3d00a3e4f920 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:55.038987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8a666a96-5b71-4342-87c6-3d00a3e4f920 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:55.040236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8a666a96-5b71-4342-87c6-3d00a3e4f920 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:55.048155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 724/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:55.086797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe478c-a053-4dc1-ab1e-63d27eca43ec None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:55.097869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aabe478c-a053-4dc1-ab1e-63d27eca43ec None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:55.101489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afec820f-2727-4fd7-a166-3bf0f67a4b53 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:55.102132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec820f-2727-4fd7-a166-3bf0f67a4b53 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:55.102421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec820f-2727-4fd7-a166-3bf0f67a4b53 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:55.102702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afec820f-2727-4fd7-a166-3bf0f67a4b53 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:55.111567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec820f-2727-4fd7-a166-3bf0f67a4b53 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:55.113035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afec820f-2727-4fd7-a166-3bf0f67a4b53 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:55.122437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 731/1455] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:49:57.284643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00026cd-cd30-4616-a350-38ef0941031a None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:57.285183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00026cd-cd30-4616-a350-38ef0941031a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:57.285183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00026cd-cd30-4616-a350-38ef0941031a None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:57.287091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 725/1456] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:57 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:57.292288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3958b4a-33a6-4adf-86b4-15ebac520dea None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:57.292737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3958b4a-33a6-4adf-86b4-15ebac520dea None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:57.292835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3958b4a-33a6-4adf-86b4-15ebac520dea None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:57.310592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a3958b4a-33a6-4adf-86b4-15ebac520dea None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:57.311791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a3958b4a-33a6-4adf-86b4-15ebac520dea None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:57.319418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 732/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:57.357081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00026cd-cd30-4616-a350-38ef0941031a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:57.368084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00026cd-cd30-4616-a350-38ef0941031a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:57.371579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c30de257-12a4-4440-95ce-158b0f13c23d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:57.372242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30de257-12a4-4440-95ce-158b0f13c23d service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:57.372524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30de257-12a4-4440-95ce-158b0f13c23d service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:57.372797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c30de257-12a4-4440-95ce-158b0f13c23d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:57.381425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30de257-12a4-4440-95ce-158b0f13c23d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:57.382838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c30de257-12a4-4440-95ce-158b0f13c23d service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:57.391962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 726/1458] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:49:59.540637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e735cf2-fc16-4be9-8fdf-f26e8dcbbbff None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:59.541180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e735cf2-fc16-4be9-8fdf-f26e8dcbbbff None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:59.541180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e735cf2-fc16-4be9-8fdf-f26e8dcbbbff None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:59.542561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 733/1459] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:49:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:49:59.547619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9506028-45aa-453e-8e29-385741764ce3 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:59.548056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9506028-45aa-453e-8e29-385741764ce3 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:59.548205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9506028-45aa-453e-8e29-385741764ce3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:59.554144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=70 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:49:59.565790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d9506028-45aa-453e-8e29-385741764ce3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:49:59.566970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d9506028-45aa-453e-8e29-385741764ce3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:49:59.575142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 727/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:49:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:49:59.611797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e735cf2-fc16-4be9-8fdf-f26e8dcbbbff None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:49:59.622381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e735cf2-fc16-4be9-8fdf-f26e8dcbbbff None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:49:59.625889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c1150e-3498-44c2-8e4f-72c34ce151e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:49:59.626547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1150e-3498-44c2-8e4f-72c34ce151e3 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:49:59.626834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1150e-3498-44c2-8e4f-72c34ce151e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:49:59.627114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c1150e-3498-44c2-8e4f-72c34ce151e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:49:59.633420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=75 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:49:59.636902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1150e-3498-44c2-8e4f-72c34ce151e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:49:59.638331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c1150e-3498-44c2-8e4f-72c34ce151e3 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:49:59.648114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 734/1461] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:49:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:01.813041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8f7e-8045-4db4-8121-7769aaabd1f7 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:01.813582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8f7e-8045-4db4-8121-7769aaabd1f7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:01.813582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b81a8f7e-8045-4db4-8121-7769aaabd1f7 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:01.814897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 728/1462] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:50:01 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:01.820115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba3c4a-2f7a-4ee2-82b9-776248c639c6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:01.820522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba3c4a-2f7a-4ee2-82b9-776248c639c6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:01.820674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba3c4a-2f7a-4ee2-82b9-776248c639c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:01.837715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fbba3c4a-2f7a-4ee2-82b9-776248c639c6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:01.838921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fbba3c4a-2f7a-4ee2-82b9-776248c639c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:01.846805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 735/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:50:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:01.884119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a8f7e-8045-4db4-8121-7769aaabd1f7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:01.894635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b81a8f7e-8045-4db4-8121-7769aaabd1f7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:01.898271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e201dd-7aae-45d0-9e39-29be9155cfd7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:01.898948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e201dd-7aae-45d0-9e39-29be9155cfd7 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:01.899235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e201dd-7aae-45d0-9e39-29be9155cfd7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:01.899514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e201dd-7aae-45d0-9e39-29be9155cfd7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:01.908536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e201dd-7aae-45d0-9e39-29be9155cfd7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:01.909971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e201dd-7aae-45d0-9e39-29be9155cfd7 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:01.919895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 729/1464] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:04.070585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbf0cc-9882-44c6-936a-c781160b5121 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:04.071150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbf0cc-9882-44c6-936a-c781160b5121 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:04.071150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abbf0cc-9882-44c6-936a-c781160b5121 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:04.072571 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 736/1465] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:50:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:04.078022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e5c44-bb18-4d4c-8888-bda34a7175aa None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:04.078459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e5c44-bb18-4d4c-8888-bda34a7175aa None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:04.078580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a88e5c44-bb18-4d4c-8888-bda34a7175aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:04.097393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a88e5c44-bb18-4d4c-8888-bda34a7175aa None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:04.098580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a88e5c44-bb18-4d4c-8888-bda34a7175aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:04.106226 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 730/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:50:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:04.142159 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbf0cc-9882-44c6-936a-c781160b5121 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:04.153427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abbf0cc-9882-44c6-936a-c781160b5121 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:04.156973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0064c3ea-2a68-4a45-8f44-382260136133 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:04.157608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064c3ea-2a68-4a45-8f44-382260136133 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:04.157897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064c3ea-2a68-4a45-8f44-382260136133 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:04.158181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0064c3ea-2a68-4a45-8f44-382260136133 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:04.166994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064c3ea-2a68-4a45-8f44-382260136133 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:04.168439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0064c3ea-2a68-4a45-8f44-382260136133 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:04.177735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 737/1467] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:06.321521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1992514-2e8f-4836-9221-771596d7edc0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:06.322091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1992514-2e8f-4836-9221-771596d7edc0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:06.322091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1992514-2e8f-4836-9221-771596d7edc0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:06.323442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 731/1468] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:50:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:06.328524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888b25-552b-450c-82ab-b28a733ce01f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:06.328936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888b25-552b-450c-82ab-b28a733ce01f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:06.329116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6888b25-552b-450c-82ab-b28a733ce01f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:06.347229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b6888b25-552b-450c-82ab-b28a733ce01f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:06.348458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b6888b25-552b-450c-82ab-b28a733ce01f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:06.356511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 738/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:50:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:06.392284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1992514-2e8f-4836-9221-771596d7edc0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:06.403461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1992514-2e8f-4836-9221-771596d7edc0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:06.406936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc08945-5815-407e-a813-5c5d35b60bca service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:06.407568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc08945-5815-407e-a813-5c5d35b60bca service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:06.407852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc08945-5815-407e-a813-5c5d35b60bca service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:06.408153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc08945-5815-407e-a813-5c5d35b60bca service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:06.416749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc08945-5815-407e-a813-5c5d35b60bca service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:06.418167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc08945-5815-407e-a813-5c5d35b60bca service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:06.427262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 732/1470] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:16.342004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=15 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:50:16.423198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=17 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:50:26.225006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74d600-88b5-476c-9a09-ae92399e5b6d None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:26.225579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74d600-88b5-476c-9a09-ae92399e5b6d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:26.225579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b74d600-88b5-476c-9a09-ae92399e5b6d None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:26.226929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 739/1471] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:26.232101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87149f5-dccb-4f7f-ac79-e891d6fa5ea6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:26.232527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87149f5-dccb-4f7f-ac79-e891d6fa5ea6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:26.232657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d87149f5-dccb-4f7f-ac79-e891d6fa5ea6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:26.249857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d87149f5-dccb-4f7f-ac79-e891d6fa5ea6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:26.251054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d87149f5-dccb-4f7f-ac79-e891d6fa5ea6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:26.259246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 733/1472] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:26.305413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db3c8-bad9-43e4-ac1c-06d8ff880276 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:26.305704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db3c8-bad9-43e4-ac1c-06d8ff880276 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:26.305989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-373db3c8-bad9-43e4-ac1c-06d8ff880276 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:26.307354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 740/1473] 10.222.0.40 () {54 vars in 962 bytes} [Thu Jun 16 11:50:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:26.312119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b8237-0796-4596-8c0c-02f0c1255927 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:26.312412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b8237-0796-4596-8c0c-02f0c1255927 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:26.312691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-124b8237-0796-4596-8c0c-02f0c1255927 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:26.330619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-124b8237-0796-4596-8c0c-02f0c1255927 None None] MFA Rules not processed for user `35d68b3bee5445849c59da135df06fea`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:26.331794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-124b8237-0796-4596-8c0c-02f0c1255927 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:26.338749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 734/1474] 10.222.0.40 () {58 vars in 1059 bytes} [Thu Jun 16 11:50:26 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:26.345941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db3c8-bad9-43e4-ac1c-06d8ff880276 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:26.356372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-373db3c8-bad9-43e4-ac1c-06d8ff880276 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:26.359829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-990c375f-675b-4dfc-a2c2-df5833a5483c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:26.360483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c375f-675b-4dfc-a2c2-df5833a5483c service glance] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:26.360769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c375f-675b-4dfc-a2c2-df5833a5483c service glance] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:26.361047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-990c375f-675b-4dfc-a2c2-df5833a5483c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:26.370283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c375f-675b-4dfc-a2c2-df5833a5483c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:26.371695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-990c375f-675b-4dfc-a2c2-df5833a5483c service glance] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:26.381595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 741/1475] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:28.591088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d2c8-a016-47b1-a220-0834144a8cd0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:28.591651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d2c8-a016-47b1-a220-0834144a8cd0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:28.591651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c349d2c8-a016-47b1-a220-0834144a8cd0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:28.593299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 735/1476] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:28.598961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b9a5d-3a0a-4a4d-a7ba-fe50bb060065 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:28.599389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b9a5d-3a0a-4a4d-a7ba-fe50bb060065 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:28.599528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e33b9a5d-3a0a-4a4d-a7ba-fe50bb060065 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:28.617363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e33b9a5d-3a0a-4a4d-a7ba-fe50bb060065 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:28.618556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e33b9a5d-3a0a-4a4d-a7ba-fe50bb060065 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:28.626308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 742/1477] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:28.649378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349d2c8-a016-47b1-a220-0834144a8cd0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:28.661957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c349d2c8-a016-47b1-a220-0834144a8cd0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:28.666320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:28.666954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:28.667236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:28.667511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:28.668274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:28.670081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:50:28.670081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:50:28.670081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:50:28.671212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afeaed22-09de-46d3-9876-570c7ce37cdf admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:28.689925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 736/1478] 10.222.0.40 () {56 vars in 1108 bytes} [Thu Jun 16 11:50:28 2022] GET /identity/v3/projects => generated 2553 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:50:30.801738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c4f86-9371-46d1-a0b7-3fe765d6372e None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:30.802274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c4f86-9371-46d1-a0b7-3fe765d6372e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:30.802274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-805c4f86-9371-46d1-a0b7-3fe765d6372e None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:30.803697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 743/1479] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:30.808745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a4f9-34b3-4e34-b205-b7e9ba6dd5ff None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:30.809174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a4f9-34b3-4e34-b205-b7e9ba6dd5ff None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:30.809305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10e6a4f9-34b3-4e34-b205-b7e9ba6dd5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:30.827403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-10e6a4f9-34b3-4e34-b205-b7e9ba6dd5ff None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:30.828650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-10e6a4f9-34b3-4e34-b205-b7e9ba6dd5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:30.836977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 737/1480] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:30.879630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c4f86-9371-46d1-a0b7-3fe765d6372e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:30.893698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-805c4f86-9371-46d1-a0b7-3fe765d6372e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:30.899246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-254bd0ca-eb48-4167-9463-5a3e68824d36 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:30.899909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bd0ca-eb48-4167-9463-5a3e68824d36 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:30.900202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bd0ca-eb48-4167-9463-5a3e68824d36 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:30.900481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-254bd0ca-eb48-4167-9463-5a3e68824d36 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:30.908575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bd0ca-eb48-4167-9463-5a3e68824d36 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:30.910136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-254bd0ca-eb48-4167-9463-5a3e68824d36 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:30.919425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 744/1481] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:33.206098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2374-09f0-4a40-80e3-db4809becb70 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:33.207144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2374-09f0-4a40-80e3-db4809becb70 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:33.207339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbc2374-09f0-4a40-80e3-db4809becb70 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:33.208748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 738/1482] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:33.213917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb9aa8-2a32-4c8b-83a8-994c5e97cb4a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:33.214335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb9aa8-2a32-4c8b-83a8-994c5e97cb4a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:33.214484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52cb9aa8-2a32-4c8b-83a8-994c5e97cb4a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:33.232734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-52cb9aa8-2a32-4c8b-83a8-994c5e97cb4a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:33.233955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-52cb9aa8-2a32-4c8b-83a8-994c5e97cb4a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:33.241706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 745/1483] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:33.285426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2374-09f0-4a40-80e3-db4809becb70 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:33.288332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcbc2374-09f0-4a40-80e3-db4809becb70 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:33.291796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30936803-1d68-4aab-889f-e8df8015d755 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:33.292455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30936803-1d68-4aab-889f-e8df8015d755 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:33.292738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30936803-1d68-4aab-889f-e8df8015d755 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:33.293010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30936803-1d68-4aab-889f-e8df8015d755 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:33.301877 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30936803-1d68-4aab-889f-e8df8015d755 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:33.303408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30936803-1d68-4aab-889f-e8df8015d755 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:33.312734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 739/1484] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:35.469644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64727cc-83a6-4b3a-b230-8b638f1b9e23 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:35.470272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64727cc-83a6-4b3a-b230-8b638f1b9e23 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:35.470272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a64727cc-83a6-4b3a-b230-8b638f1b9e23 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:35.471599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 746/1485] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:35.477379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a0c8-66c1-47c0-a2be-d82e42b21029 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:35.477790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a0c8-66c1-47c0-a2be-d82e42b21029 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:35.477942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abc9a0c8-66c1-47c0-a2be-d82e42b21029 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:35.495133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-abc9a0c8-66c1-47c0-a2be-d82e42b21029 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:35.496361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-abc9a0c8-66c1-47c0-a2be-d82e42b21029 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:35.504126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 740/1486] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:35.542288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64727cc-83a6-4b3a-b230-8b638f1b9e23 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:35.552985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a64727cc-83a6-4b3a-b230-8b638f1b9e23 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:35.556522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e3a97f3-8321-4f73-9c8f-e0f4d3509b01 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:35.557168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a97f3-8321-4f73-9c8f-e0f4d3509b01 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:35.557460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a97f3-8321-4f73-9c8f-e0f4d3509b01 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:35.557887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e3a97f3-8321-4f73-9c8f-e0f4d3509b01 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:35.569768 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a97f3-8321-4f73-9c8f-e0f4d3509b01 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:35.571196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e3a97f3-8321-4f73-9c8f-e0f4d3509b01 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:35.580758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 747/1487] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:37.788451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ef7f-3461-495b-9059-88fa8aeda583 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:37.794041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ef7f-3461-495b-9059-88fa8aeda583 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:37.794041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bb3ef7f-3461-495b-9059-88fa8aeda583 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:37.794041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 741/1488] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:37.795496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c5de-15fc-49de-ba80-d93f32314be7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:37.795944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c5de-15fc-49de-ba80-d93f32314be7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:37.796109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51b8c5de-15fc-49de-ba80-d93f32314be7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:37.813910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-51b8c5de-15fc-49de-ba80-d93f32314be7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:37.815149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-51b8c5de-15fc-49de-ba80-d93f32314be7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:37.823694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 748/1489] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:37.864075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ef7f-3461-495b-9059-88fa8aeda583 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:37.874702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bb3ef7f-3461-495b-9059-88fa8aeda583 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:37.878811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e86e5e7f-c518-4f4e-9239-a43a490d2de5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:37.879460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e5e7f-c518-4f4e-9239-a43a490d2de5 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:37.879741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e5e7f-c518-4f4e-9239-a43a490d2de5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:37.880052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e86e5e7f-c518-4f4e-9239-a43a490d2de5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:37.888690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e5e7f-c518-4f4e-9239-a43a490d2de5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:37.890109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e86e5e7f-c518-4f4e-9239-a43a490d2de5 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:37.899488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 742/1490] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:40.405021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf3d0e-bf69-4353-bb0d-70b59d858196 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:40.405586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf3d0e-bf69-4353-bb0d-70b59d858196 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:40.405586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaf3d0e-bf69-4353-bb0d-70b59d858196 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:40.406970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 749/1491] 10.222.0.40 () {54 vars in 926 bytes} [Thu Jun 16 11:50:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:50:40.412150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332e557-7966-4296-93e2-eb70e6e34b38 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:40.412533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332e557-7966-4296-93e2-eb70e6e34b38 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:40.412714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4332e557-7966-4296-93e2-eb70e6e34b38 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:40.430263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4332e557-7966-4296-93e2-eb70e6e34b38 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:50:40.431452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4332e557-7966-4296-93e2-eb70e6e34b38 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:50:40.439142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 743/1492] 10.222.0.40 () {58 vars in 1023 bytes} [Thu Jun 16 11:50:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:50:40.477837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf3d0e-bf69-4353-bb0d-70b59d858196 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:50:40.488413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcaf3d0e-bf69-4353-bb0d-70b59d858196 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:50:40.491990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52887054-faed-43ad-a09e-6b4a3602c569 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:50:40.492631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52887054-faed-43ad-a09e-6b4a3602c569 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:50:40.492920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52887054-faed-43ad-a09e-6b4a3602c569 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:50:40.493199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52887054-faed-43ad-a09e-6b4a3602c569 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:50:40.501940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52887054-faed-43ad-a09e-6b4a3602c569 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:50:40.503891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52887054-faed-43ad-a09e-6b4a3602c569 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:50:40.513195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 750/1493] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:50:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:50:50.425254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=33 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:50:50.509061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=33 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:52:21.034272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.034804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.034914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.052330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:52:21.053523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:21.061195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 744/1494] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:52:21.070486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.070971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.071070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.088866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:52:21.090061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:21.098734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 751/1495] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:52:21.103151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.113221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdff881c-b991-49aa-8c21-190965f1ac5e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.117685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.118331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.118613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.118890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.119612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:52:21.119944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.120944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.120944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:52:21.120944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:52:21.121817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.133524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 745/1496] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:52:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:52:21.141225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.150245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f82957-8df2-42a4-8d9c-98a3b2133b31 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.154898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.155548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.155835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.156141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.156832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.158610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.158610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:52:21.158610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:52:21.159799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.254665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 752/1497] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:52:21 2022] POST /identity/v3/projects => generated 377 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:52:21.258900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.268808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2454ea55-1cb6-4b3b-881f-c004080a2c7c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.323986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.324653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.325104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.325390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.326071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.327107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.327107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:52:21.327107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:52:21.328344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.426286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 746/1498] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:52:21 2022] POST /identity/v3/users => generated 289 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:52:21.430410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.440694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77c35eb6-4db6-4065-b699-de91cb141e0d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.445263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:52:21.452460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:52:21.453061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.461455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 753/1499] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:52:21.465540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.476120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519d91bc-dedc-454e-af8d-f18e6ddeeda6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.480808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.481498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.481784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.482063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.482787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.483792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.483792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:52:21.483792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:52:21.484725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7741c03-e95a-4782-b1fb-215922bc8bc0 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.498535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 747/1500] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:52:21 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:52:21.506406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.513101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bb9e879-bbee-458e-9561-8e6e19348aec admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.517816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.518734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.519013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.519292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] PATH_INFO: `/v3/projects/d589cc9033824f84bbeebc5d582bfea6/users/c92e7236ad3e40528b7f74cf3d2b1915/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.534407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] RBAC: Authorizing `identity:create_grant(project_id=d589cc9033824f84bbeebc5d582bfea6, user_id=c92e7236ad3e40528b7f74cf3d2b1915, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.535430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.535430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:52:21.535430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:52:21.537393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d7f2395-c887-48cb-9021-3f485150571a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.602769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 754/1501] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:52:21 2022] PUT /identity/v3/projects/d589cc9033824f84bbeebc5d582bfea6/users/c92e7236ad3e40528b7f74cf3d2b1915/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 100 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:52:21.614774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5640f8-8a4e-4993-b39d-55b3e00a0d81 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.615071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5640f8-8a4e-4993-b39d-55b3e00a0d81 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.615362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5640f8-8a4e-4993-b39d-55b3e00a0d81 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.641377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4f5640f8-8a4e-4993-b39d-55b3e00a0d81 None None] MFA Rules not processed for user `c92e7236ad3e40528b7f74cf3d2b1915`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:52:21.642667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4f5640f8-8a4e-4993-b39d-55b3e00a0d81 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:21.705553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 748/1502] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:52:21.718831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742244ab-f378-4440-a6bc-a27cfc4aa89c None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.719150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742244ab-f378-4440-a6bc-a27cfc4aa89c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.719432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-742244ab-f378-4440-a6bc-a27cfc4aa89c None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.721380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 755/1503] 10.222.0.40 () {54 vars in 963 bytes} [Thu Jun 16 11:52:21 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:52:21.726009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79206db5-c173-4123-8890-75b6ff75ef1f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.726305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79206db5-c173-4123-8890-75b6ff75ef1f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.726587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79206db5-c173-4123-8890-75b6ff75ef1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.745777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-79206db5-c173-4123-8890-75b6ff75ef1f None None] MFA Rules not processed for user `72c24cb67566467dbdf65cfd44a59274`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:52:21.746970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-79206db5-c173-4123-8890-75b6ff75ef1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:21.789626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:52:21.796877 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 749/1504] 10.222.0.40 () {58 vars in 1060 bytes} [Thu Jun 16 11:52:21 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:52:21.804367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742244ab-f378-4440-a6bc-a27cfc4aa89c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.814857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-742244ab-f378-4440-a6bc-a27cfc4aa89c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.818354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.818992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.819279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.819559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.827658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.831314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:21.838241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 756/1505] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:52:21 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:52:21.879794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0de57-68c3-4027-925d-7315894e5571 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.880123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0de57-68c3-4027-925d-7315894e5571 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.880407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ee0de57-68c3-4027-925d-7315894e5571 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.881683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 750/1506] 10.222.0.40 () {52 vars in 829 bytes} [Thu Jun 16 11:52:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:52:21.885308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:21.946508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c5c88d-4d28-4de9-9994-ab3b0496ef32 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:21.950984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:21.951851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:21.952172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:21.952450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] PATH_INFO: `/v3/users/c92e7236ad3e40528b7f74cf3d2b1915` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:21.953616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] RBAC: Authorizing `identity:delete_user(user_id=c92e7236ad3e40528b7f74cf3d2b1915)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:21.954638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:21.954638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:52:21.954638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:52:21.956343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:22.140896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c92e7236ad3e40528b7f74cf3d2b1915'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:22.141591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c92e7236ad3e40528b7f74cf3d2b1915'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:22.196854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c92e7236ad3e40528b7f74cf3d2b1915'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:22.209387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c92e7236ad3e40528b7f74cf3d2b1915'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:22.226425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=88,keystone:INSERT=3,keystone:DELETE=9 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:52:22.228953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c92e7236ad3e40528b7f74cf3d2b1915'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:22.234962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 757/1507] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:52:21 2022] DELETE /identity/v3/users/c92e7236ad3e40528b7f74cf3d2b1915 => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:52:22.240279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:22.240755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:22.240851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:22.257707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:52:22.258923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:22.315930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 751/1508] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:52:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:52:22.325514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:22.326895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:22.384273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c24a5bca-0162-4d9a-87f2-ceb0b167385f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:22.387799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74e93de8-abdd-406c-b2e6-0016ba474353 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:22.388468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e93de8-abdd-406c-b2e6-0016ba474353 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:22.388752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e93de8-abdd-406c-b2e6-0016ba474353 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:22.389449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74e93de8-abdd-406c-b2e6-0016ba474353 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:22.398263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e93de8-abdd-406c-b2e6-0016ba474353 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:22.399642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74e93de8-abdd-406c-b2e6-0016ba474353 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:22.409467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 758/1509] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:52:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:52:22.873321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:52:22.874938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:52:22.887290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad93add-fe3b-41ec-af6f-ebee38c615ad None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:52:22.891842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:52:22.892730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:52:22.893015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:52:22.893290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] PATH_INFO: `/v3/projects/d589cc9033824f84bbeebc5d582bfea6` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:52:22.895141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] RBAC: Authorizing `identity:delete_project(project_id=d589cc9033824f84bbeebc5d582bfea6)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:52:22.896190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:52:22.896190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:52:22.896190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:52:22.897525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:52:23.001690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd589cc9033824f84bbeebc5d582bfea6', 'request_id': 'req-1d56eec7-a5cf-4042-9719-85c80dad713a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:23.008103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd589cc9033824f84bbeebc5d582bfea6', 'request_id': 'req-1d56eec7-a5cf-4042-9719-85c80dad713a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:23.092178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd589cc9033824f84bbeebc5d582bfea6', 'request_id': 'req-1d56eec7-a5cf-4042-9719-85c80dad713a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:23.110455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] The token cache is being invalidate because project d589cc9033824f84bbeebc5d582bfea6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:52:23.110743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:23.111481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1d56eec7-a5cf-4042-9719-85c80dad713a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:52:23.113044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 752/1510] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:52:22 2022] DELETE /identity/v3/projects/d589cc9033824f84bbeebc5d582bfea6 => generated 0 bytes in 240 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:52:32.404922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=1,keystone:SELECT=28 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:52:33.107480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=55,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:53:59.512338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9f1b-2a86-470e-ac8e-5f63a6a833ff None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:53:59.512934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9f1b-2a86-470e-ac8e-5f63a6a833ff None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:53:59.512934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57cf9f1b-2a86-470e-ac8e-5f63a6a833ff None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:53:59.514350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 759/1511] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:53:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:53:59.519610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efae69-2b75-4a19-9017-94fc858fca1c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:53:59.520041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efae69-2b75-4a19-9017-94fc858fca1c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:53:59.520202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6efae69-2b75-4a19-9017-94fc858fca1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:53:59.538056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a6efae69-2b75-4a19-9017-94fc858fca1c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:53:59.539261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a6efae69-2b75-4a19-9017-94fc858fca1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:53:59.595742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 753/1512] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:53:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:53:59.769642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9f1b-2a86-470e-ac8e-5f63a6a833ff None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:53:59.769642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-57cf9f1b-2a86-470e-ac8e-5f63a6a833ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:53:59.839107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57cf9f1b-2a86-470e-ac8e-5f63a6a833ff None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:53:59.848761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e9adfd-6d4c-4251-a9c8-5ddc09e3b3a9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:53:59.849471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9adfd-6d4c-4251-a9c8-5ddc09e3b3a9 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:53:59.849760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9adfd-6d4c-4251-a9c8-5ddc09e3b3a9 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:53:59.850033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e9adfd-6d4c-4251-a9c8-5ddc09e3b3a9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:53:59.858034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9adfd-6d4c-4251-a9c8-5ddc09e3b3a9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:53:59.859629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e9adfd-6d4c-4251-a9c8-5ddc09e3b3a9 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:53:59.869460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 760/1513] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:53:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:00.740598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7176-fa24-46ba-9908-9a197bf84da0 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:00.741102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7176-fa24-46ba-9908-9a197bf84da0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:00.741204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebd7176-fa24-46ba-9908-9a197bf84da0 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:00.742536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 754/1514] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:54:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:00.747215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6c64-bf3c-4342-835c-fad320c21a6f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:00.747660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6c64-bf3c-4342-835c-fad320c21a6f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:00.747764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cd6c64-bf3c-4342-835c-fad320c21a6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:00.765570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-03cd6c64-bf3c-4342-835c-fad320c21a6f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:00.766767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-03cd6c64-bf3c-4342-835c-fad320c21a6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:00.774793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 761/1515] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:54:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:54:00.929853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7176-fa24-46ba-9908-9a197bf84da0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:00.944595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebd7176-fa24-46ba-9908-9a197bf84da0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:00.949978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d34ad452-639a-4ec2-a568-21b163507dff service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:00.950668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ad452-639a-4ec2-a568-21b163507dff service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:00.950954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ad452-639a-4ec2-a568-21b163507dff service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:00.951233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d34ad452-639a-4ec2-a568-21b163507dff service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:00.959658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ad452-639a-4ec2-a568-21b163507dff service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:00.961290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d34ad452-639a-4ec2-a568-21b163507dff service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:00.971585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 755/1516] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:01.253866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748043c-480b-40b9-a5c1-0456f4a3e381 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:01.254158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748043c-480b-40b9-a5c1-0456f4a3e381 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:01.254444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1748043c-480b-40b9-a5c1-0456f4a3e381 None None] PATH_INFO: `/v3` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:01.255768 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 762/1517] 10.222.0.13 () {54 vars in 925 bytes} [Thu Jun 16 11:54:01 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:54:01.262203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d128bd-c29d-44e5-a1ae-456850cc7cd8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:01.262501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d128bd-c29d-44e5-a1ae-456850cc7cd8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:01.262789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d128bd-c29d-44e5-a1ae-456850cc7cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:01.281051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b9d128bd-c29d-44e5-a1ae-456850cc7cd8 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:01.282270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b9d128bd-c29d-44e5-a1ae-456850cc7cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:01.343527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 756/1518] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 16 11:54:01 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:05.256299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fe3c5-1b88-434b-9e44-c776c83e42f3 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:05.256848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fe3c5-1b88-434b-9e44-c776c83e42f3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:05.256848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-240fe3c5-1b88-434b-9e44-c776c83e42f3 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:05.258141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 763/1519] 10.222.0.40 () {54 vars in 960 bytes} [Thu Jun 16 11:54:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:05.263273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e9ac-1d9d-468d-9dae-e4257e485cc6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:05.263682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e9ac-1d9d-468d-9dae-e4257e485cc6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:05.263834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ca0e9ac-1d9d-468d-9dae-e4257e485cc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:05.281449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2ca0e9ac-1d9d-468d-9dae-e4257e485cc6 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:05.283119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ca0e9ac-1d9d-468d-9dae-e4257e485cc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:05.291459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 757/1520] 10.222.0.40 () {58 vars in 1057 bytes} [Thu Jun 16 11:54:05 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:05.303458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fe3c5-1b88-434b-9e44-c776c83e42f3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:05.316872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-240fe3c5-1b88-434b-9e44-c776c83e42f3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:05.321808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b1e98c-6b36-454a-85d4-06e5e3a2ac20 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:05.322441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1e98c-6b36-454a-85d4-06e5e3a2ac20 service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:05.322730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1e98c-6b36-454a-85d4-06e5e3a2ac20 service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:05.323013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b1e98c-6b36-454a-85d4-06e5e3a2ac20 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:05.331117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1e98c-6b36-454a-85d4-06e5e3a2ac20 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:05.332774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49b1e98c-6b36-454a-85d4-06e5e3a2ac20 service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:05.342284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 764/1521] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:05 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:12.245813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7289da-7ae9-4254-a1d5-9ebb05ef953e None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:12.246396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7289da-7ae9-4254-a1d5-9ebb05ef953e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:12.246396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b7289da-7ae9-4254-a1d5-9ebb05ef953e None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:12.247763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 758/1522] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:54:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:12.252741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48265b10-93bb-4056-873d-086b402eb725 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:12.253156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48265b10-93bb-4056-873d-086b402eb725 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:12.253305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48265b10-93bb-4056-873d-086b402eb725 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:12.270738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-48265b10-93bb-4056-873d-086b402eb725 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:12.271962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-48265b10-93bb-4056-873d-086b402eb725 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:12.279636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 765/1523] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:54:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:54:12.436263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7289da-7ae9-4254-a1d5-9ebb05ef953e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:12.450390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b7289da-7ae9-4254-a1d5-9ebb05ef953e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:12.456393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6476b24e-e90b-4e92-a980-cc7013813fa5 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:12.457026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476b24e-e90b-4e92-a980-cc7013813fa5 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:12.457315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476b24e-e90b-4e92-a980-cc7013813fa5 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:12.457593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6476b24e-e90b-4e92-a980-cc7013813fa5 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:12.465786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476b24e-e90b-4e92-a980-cc7013813fa5 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:12.467368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6476b24e-e90b-4e92-a980-cc7013813fa5 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:12.477671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 759/1524] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:13.348162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439184e9-39b7-4d88-824e-d63742ce64d9 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:13.348726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439184e9-39b7-4d88-824e-d63742ce64d9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:13.348726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-439184e9-39b7-4d88-824e-d63742ce64d9 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:13.350116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 766/1525] 127.0.0.1 () {54 vars in 901 bytes} [Thu Jun 16 11:54:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:13.354898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9533-4f7b-418b-bf52-1aeb564ba15f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:13.355279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9533-4f7b-418b-bf52-1aeb564ba15f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:13.355471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75dc9533-4f7b-418b-bf52-1aeb564ba15f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:13.375236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-75dc9533-4f7b-418b-bf52-1aeb564ba15f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:13.377221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-75dc9533-4f7b-418b-bf52-1aeb564ba15f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:13.385613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 760/1526] 127.0.0.1 () {58 vars in 998 bytes} [Thu Jun 16 11:54:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:13.538673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439184e9-39b7-4d88-824e-d63742ce64d9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:13.553547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-439184e9-39b7-4d88-824e-d63742ce64d9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:13.559095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e609ac31-bd3f-40f1-af21-e9837dd1b5bb service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:13.559734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ac31-bd3f-40f1-af21-e9837dd1b5bb service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:13.560052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ac31-bd3f-40f1-af21-e9837dd1b5bb service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:13.560334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e609ac31-bd3f-40f1-af21-e9837dd1b5bb service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:13.570940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ac31-bd3f-40f1-af21-e9837dd1b5bb service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:13.572597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e609ac31-bd3f-40f1-af21-e9837dd1b5bb service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:13.581868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 767/1527] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:17.531429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539fde0-1e61-414e-ad31-fc7181a88013 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:17.531990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539fde0-1e61-414e-ad31-fc7181a88013 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:17.531990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4539fde0-1e61-414e-ad31-fc7181a88013 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:17.533820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 761/1528] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:54:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:17.539046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e41f-8d65-4a64-ba44-99e77197feb6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:17.539473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e41f-8d65-4a64-ba44-99e77197feb6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:17.539611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0b8e41f-8d65-4a64-ba44-99e77197feb6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:17.559824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-f0b8e41f-8d65-4a64-ba44-99e77197feb6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:17.561078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f0b8e41f-8d65-4a64-ba44-99e77197feb6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:17.568831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 768/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:54:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:17.606112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539fde0-1e61-414e-ad31-fc7181a88013 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:17.607815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4539fde0-1e61-414e-ad31-fc7181a88013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:17.663199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4539fde0-1e61-414e-ad31-fc7181a88013 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:17.666688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08b93140-b2a7-4150-a67f-b4e454907a80 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:17.667324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b93140-b2a7-4150-a67f-b4e454907a80 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:17.667608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b93140-b2a7-4150-a67f-b4e454907a80 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:17.667906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08b93140-b2a7-4150-a67f-b4e454907a80 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:17.676797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b93140-b2a7-4150-a67f-b4e454907a80 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:17.678219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08b93140-b2a7-4150-a67f-b4e454907a80 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:17.686515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:54:17.691456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 762/1530] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:19.790889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2d20f-fbaa-4e48-ba98-c8d495d2e758 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:19.791442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2d20f-fbaa-4e48-ba98-c8d495d2e758 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:19.791442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5b2d20f-fbaa-4e48-ba98-c8d495d2e758 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:19.792841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 769/1531] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:54:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:19.798476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe928-2ff2-415e-9bd3-4ac4bd73ad42 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:19.798890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe928-2ff2-415e-9bd3-4ac4bd73ad42 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:19.799021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0efe928-2ff2-415e-9bd3-4ac4bd73ad42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:19.817308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f0efe928-2ff2-415e-9bd3-4ac4bd73ad42 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:19.818527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f0efe928-2ff2-415e-9bd3-4ac4bd73ad42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:19.826332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 763/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:54:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:19.861462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2d20f-fbaa-4e48-ba98-c8d495d2e758 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:19.872130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5b2d20f-fbaa-4e48-ba98-c8d495d2e758 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:19.875645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87d0e941-13dc-48d5-a6fd-65ed7581dca9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:19.876929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0e941-13dc-48d5-a6fd-65ed7581dca9 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:19.877222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0e941-13dc-48d5-a6fd-65ed7581dca9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:19.877503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87d0e941-13dc-48d5-a6fd-65ed7581dca9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:19.886226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0e941-13dc-48d5-a6fd-65ed7581dca9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:19.887656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87d0e941-13dc-48d5-a6fd-65ed7581dca9 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:19.896911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 770/1533] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:54:23.563259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fcddb-4683-4974-89b8-d55867deae08 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:23.563808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fcddb-4683-4974-89b8-d55867deae08 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:23.563808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092fcddb-4683-4974-89b8-d55867deae08 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:23.565238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 764/1534] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:54:23 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:23.570609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb456bc-04db-4c30-b44f-6fe425ac8427 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:23.571065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb456bc-04db-4c30-b44f-6fe425ac8427 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:23.571173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb456bc-04db-4c30-b44f-6fe425ac8427 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:23.577531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=62 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:54:23.589819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1cb456bc-04db-4c30-b44f-6fe425ac8427 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:23.591009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1cb456bc-04db-4c30-b44f-6fe425ac8427 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:23.599602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 771/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:54:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:23.865811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fcddb-4683-4974-89b8-d55867deae08 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:23.867533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-092fcddb-4683-4974-89b8-d55867deae08 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:23.924383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092fcddb-4683-4974-89b8-d55867deae08 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:23.927959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ddfa1e7-066d-46fd-87df-40f1bac29381 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:23.928607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa1e7-066d-46fd-87df-40f1bac29381 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:23.928897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa1e7-066d-46fd-87df-40f1bac29381 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:23.929196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ddfa1e7-066d-46fd-87df-40f1bac29381 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:23.938524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfa1e7-066d-46fd-87df-40f1bac29381 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:23.939983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ddfa1e7-066d-46fd-87df-40f1bac29381 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:23.949272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 765/1536] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:26.076115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e451b-9dd6-488b-8dbf-e58102fdf518 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:26.076700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e451b-9dd6-488b-8dbf-e58102fdf518 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:26.076700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e49e451b-9dd6-488b-8dbf-e58102fdf518 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:26.078047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 772/1537] 127.0.0.1 () {54 vars in 916 bytes} [Thu Jun 16 11:54:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:54:26.083644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e743a4-7e20-4b05-93a6-17ba7238b888 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:26.084071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e743a4-7e20-4b05-93a6-17ba7238b888 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:26.084244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e743a4-7e20-4b05-93a6-17ba7238b888 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:26.102787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-60e743a4-7e20-4b05-93a6-17ba7238b888 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:54:26.104026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-60e743a4-7e20-4b05-93a6-17ba7238b888 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:26.111740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 766/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Thu Jun 16 11:54:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:54:26.373779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e451b-9dd6-488b-8dbf-e58102fdf518 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:54:26.375988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e49e451b-9dd6-488b-8dbf-e58102fdf518 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:54:26.388671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e49e451b-9dd6-488b-8dbf-e58102fdf518 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:54:26.392770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2665b069-5db7-4b70-a72e-6f0e5cf99622 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:54:26.393402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665b069-5db7-4b70-a72e-6f0e5cf99622 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:54:26.393686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665b069-5db7-4b70-a72e-6f0e5cf99622 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:54:26.393960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2665b069-5db7-4b70-a72e-6f0e5cf99622 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:54:26.402605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665b069-5db7-4b70-a72e-6f0e5cf99622 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:54:26.404035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2665b069-5db7-4b70-a72e-6f0e5cf99622 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:54:26.413212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 773/1539] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:54:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:54:36.097512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=32 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:54:36.409178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=8 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:53.283906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.284543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.284543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.310583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:53.312305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:53.326504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 767/1540] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:55:53.338018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.338623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.338623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.364970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:53.366251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:53.379469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 774/1541] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 43 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:53.384758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.399724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82b6a37b-3833-4b93-b930-002a0793345c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:53.408635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:53.409663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.410180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.410892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.412125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:53.412690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:53.414556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c221dcc2-7e70-460c-9a93-cb01e4e5103a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:53.438921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 768/1542] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:53.447474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.457133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-201e9da5-8e02-4407-af55-3e332ac79ea7 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:53.466408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:53.467098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.467384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.467660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.468935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:53.471070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:53.521878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a7a1f-edf1-4d91-90ce-2d5758c78a20 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.522949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a7a1f-edf1-4d91-90ce-2d5758c78a20 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.523457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-937a7a1f-edf1-4d91-90ce-2d5758c78a20 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.546714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-937a7a1f-edf1-4d91-90ce-2d5758c78a20 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:53.548323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-937a7a1f-edf1-4d91-90ce-2d5758c78a20 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:53.560050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 769/1543] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:53.570113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.573164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.573164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.602527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:53.604551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:53.609473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 775/1544] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/projects => generated 371 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:53.613814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.716241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 770/1545] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:55:53.722134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.738166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0783bd3e-71a7-45ec-865f-3a7b832ca301 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:53.739802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-830a6d2e-0c46-41d5-a484-9c2eb1079112 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:53.746170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:53.746672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:53.746869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.747161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.747443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.748053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.748210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:53.749876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.750311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.751242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:53.757301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:55:53.757301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:55:53.757301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:55:53.757301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:53.758039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:53.759963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70d70441-f052-43b5-9e08-b00a06a9de21 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:53.785821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 771/1546] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:55:53.787363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.787704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.788028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.825297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:53.826753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:53.841632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 772/1547] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 55 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:53.842625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.852199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 776/1548] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:53.861007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a11303bb-996e-4a04-8518-22a906c76cf8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:53.862981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.871955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:53.872650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.872940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.873353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.874065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:53.875667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:55:53.875667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:55:53.875667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:55:53.876919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:53.884538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebdb91e-bf91-4add-a2ba-9b0c0ecb6b42 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:53.897117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94312de7-29ea-428f-9599-1dbeb87a8a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:53.897820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94312de7-29ea-428f-9599-1dbeb87a8a51 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.898108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94312de7-29ea-428f-9599-1dbeb87a8a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.898385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94312de7-29ea-428f-9599-1dbeb87a8a51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.899190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94312de7-29ea-428f-9599-1dbeb87a8a51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:53.900833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94312de7-29ea-428f-9599-1dbeb87a8a51 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:53.924061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 777/1549] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:55:53.925882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ae319-2c18-4db2-85ce-8a561016acc9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.926403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ae319-2c18-4db2-85ce-8a561016acc9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.926999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00ae319-2c18-4db2-85ce-8a561016acc9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:53.960926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e00ae319-2c18-4db2-85ce-8a561016acc9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:53.962516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e00ae319-2c18-4db2-85ce-8a561016acc9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:53.982335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 778/1550] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 57 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:53.984943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 773/1551] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/projects => generated 393 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:53.985232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56670-6680-4733-811c-41ab5aa257ae None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:53.992320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56670-6680-4733-811c-41ab5aa257ae None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:53.992467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:53.992583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a56670-6680-4733-811c-41ab5aa257ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.018850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-60a56670-6680-4733-811c-41ab5aa257ae None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.020162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-60a56670-6680-4733-811c-41ab5aa257ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.071013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0ae2fc-6e84-4fee-8ff8-5057255b17df admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.078306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.078927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.079212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.079483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.080242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.081854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.087315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 779/1552] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.088445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.088734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.089013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.099285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 774/1553] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:53 2022] GET /identity/v3/roles => generated 1420 bytes in 114 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:54.100689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.107771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.109043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.112295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23e9544d-1024-4456-a5f7-c1c0cf5cd884 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.116996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.123915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.124092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 780/1554] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.124190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.124190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.124903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.125072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.125072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.138543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d89fca6-71c8-4ec2-aeaa-8ee90ad2fa7d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.144541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.144953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.145243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.145523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.146289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:54.146603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.147608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:55:54.147608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:55:54.147608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:55:54.148548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07cd058a-52bc-4dde-aa05-82a44c76c426 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.160357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 781/1555] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:54.161534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97611e0-da76-4e60-bafe-e64ffff7885c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.161826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97611e0-da76-4e60-bafe-e64ffff7885c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.162109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97611e0-da76-4e60-bafe-e64ffff7885c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.178989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a97611e0-da76-4e60-bafe-e64ffff7885c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.180218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a97611e0-da76-4e60-bafe-e64ffff7885c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.188540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 782/1556] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.189698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdca80-a351-4b76-b9e8-2675ceca58ce None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.189987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdca80-a351-4b76-b9e8-2675ceca58ce None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.190268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fffdca80-a351-4b76-b9e8-2675ceca58ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.207983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fffdca80-a351-4b76-b9e8-2675ceca58ce None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.209156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fffdca80-a351-4b76-b9e8-2675ceca58ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.216931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 783/1557] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.218126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb4909-5c27-472d-a49d-c96989ae54f0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.218668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb4909-5c27-472d-a49d-c96989ae54f0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.218949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcbb4909-5c27-472d-a49d-c96989ae54f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.236536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fcbb4909-5c27-472d-a49d-c96989ae54f0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.237756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fcbb4909-5c27-472d-a49d-c96989ae54f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.244488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 775/1558] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/users => generated 297 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:54.245475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.245898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 784/1559] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.247129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d3af4-dd3f-4e02-8e20-69220e22edd3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.247413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d3af4-dd3f-4e02-8e20-69220e22edd3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.248001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-035d3af4-dd3f-4e02-8e20-69220e22edd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.255420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:54.258647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e4efaaa-c5ed-4900-a397-cfc8c04574aa admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.263327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.264254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.264543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.264881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] PATH_INFO: `/v3/projects/e8013c8355144c5799d338a4065584f5/users/fe76d1513055439987cff3d6819b2639/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.265937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-035d3af4-dd3f-4e02-8e20-69220e22edd3 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.267144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-035d3af4-dd3f-4e02-8e20-69220e22edd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.273557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:54.275213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 785/1560] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.276453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.282108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.282108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.282311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e8013c8355144c5799d338a4065584f5, user_id=fe76d1513055439987cff3d6819b2639, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.282311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:55:54.282311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:55:54.282311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:55:54.283358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8dfa1a8-7fc3-44ea-8a8f-94b32be1a543 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.295213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.296943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.304779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 786/1561] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.305757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.315594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c52f6b-0cb6-4563-bfb5-4c5084591c13 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.320310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd2e3f3e-474e-4721-8c70-8da2c2d9f40f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.320943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e3f3e-474e-4721-8c70-8da2c2d9f40f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.321227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e3f3e-474e-4721-8c70-8da2c2d9f40f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.321498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd2e3f3e-474e-4721-8c70-8da2c2d9f40f admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.322256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e3f3e-474e-4721-8c70-8da2c2d9f40f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.324187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd2e3f3e-474e-4721-8c70-8da2c2d9f40f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.373064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 776/1562] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:54 2022] PUT /identity/v3/projects/e8013c8355144c5799d338a4065584f5/users/fe76d1513055439987cff3d6819b2639/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 128 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:54.375071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.375430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.375776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.392615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.393820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.433645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 787/1563] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:54.434929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.435413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.435532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.453066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 777/1564] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.453345 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.454134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.454624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.463572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b91eddfa-5f79-4258-80d7-7395e6d08cb2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.516188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 788/1565] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.517115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.519792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.520506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.520790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.521067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.521798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.523401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.526602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23b6f600-03eb-4f08-9ae5-7866c8dba566 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.531132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.531805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.532122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.532400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.533144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:54.536176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 778/1566] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:54.536319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.536422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.536513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-531f36ac-3a9b-4181-bffa-927e77a8b5e9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.545113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512713ac-512c-47c2-9b35-5781aa47192c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.550190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.550851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.550972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 789/1567] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:54.551130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.551405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.552188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.552310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:54.552502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.552623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.552829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.554110 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb5a97a-61d5-416b-8d0d-9c15375777b1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.570078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 779/1568] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:54.582047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.582047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.582047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.582328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.582328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.582328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 790/1569] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.582328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.591213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1801ecc7-c2f7-43da-b52a-ef67806beac9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.596485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.597170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.597456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.597733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.598041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] MFA Rules not processed for user `fe76d1513055439987cff3d6819b2639`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:54.598421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.599249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:54.600462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.659057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:54.662587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 780/1570] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:54.663571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.675547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-458fbab8-0b91-4f87-9758-328a4861afd6 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.679597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.680362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.680638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.680915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.681640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:54.681947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.683944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.697116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 781/1571] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:54.698132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.708473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77475f25-82ca-4644-99b7-76137c8cf717 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.713134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.713780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.714062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.714337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.715104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:54.715416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.719275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 791/1572] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/users => generated 288 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:54.719275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.719492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.729356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a17918e-c2bf-4de1-abf7-6a0952e6b0d6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.738221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 782/1573] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:54.738221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.738462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.738462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.738462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.738462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.738855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.738855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.742410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-631ce640-9ff4-402a-8af7-3acb1b9048a9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.747185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.747832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.748151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.748474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.748575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 792/1574] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:54.749610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.752192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.754146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.759092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccefd136-b793-4d88-acb3-3cc8771fcdc6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.765379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1ec1ed7-15a5-45c3-8a84-f04b3f0574b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.765379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec1ed7-15a5-45c3-8a84-f04b3f0574b2 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.765379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec1ed7-15a5-45c3-8a84-f04b3f0574b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.765379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1ec1ed7-15a5-45c3-8a84-f04b3f0574b2 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.765802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec1ed7-15a5-45c3-8a84-f04b3f0574b2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.767347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1ec1ed7-15a5-45c3-8a84-f04b3f0574b2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.853971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 783/1575] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/projects => generated 379 bytes in 122 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:54.854586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.867530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 793/1576] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/projects => generated 367 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:54.867530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a32f9-b073-494e-a41a-7da85a7c8682 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.867530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a32f9-b073-494e-a41a-7da85a7c8682 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.867530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b3a32f9-b073-494e-a41a-7da85a7c8682 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.868007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 794/1577] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 11:55:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:55:54.868666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a32f9-b073-494e-a41a-7da85a7c8682 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.916274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baea170d-ed09-4fdb-97ea-1b379c44e563 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.932427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b3a32f9-b073-494e-a41a-7da85a7c8682 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:54.936969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.937634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.937922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.938203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.938891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.940909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.944126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:54.971753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:54.972459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:54.972743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:54.973020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:54.973795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:55:54.974102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:54.976063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:54.988598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 784/1578] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:55:54.989557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:54.998707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7bec549-2db8-490a-b90d-83db43c2bfc3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.003253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.003769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.004064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.004345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.005041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.006583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.016626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 785/1579] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:54 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:55.017417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.026543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84ee7b7-ac2c-4792-b296-0f98b7ad7ed8 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.026878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 795/1580] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:54 2022] POST /identity/v3/projects => generated 367 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.027998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.066481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:55.082646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.083314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.083604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.083896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.084594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.086541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.089221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3b6f62-3436-4feb-98e2-9b58888114cd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.095176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.095594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.095934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.096217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] PATH_INFO: `/v3/projects/ce5409289ccf4aad8d62b13a72d0009e/users/6cdd0a7f3f12414ea85ebffc9d531773/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.111622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ce5409289ccf4aad8d62b13a72d0009e, user_id=6cdd0a7f3f12414ea85ebffc9d531773, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.114380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.207964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 786/1581] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/projects => generated 387 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.208911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.230090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 796/1582] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:55 2022] PUT /identity/v3/projects/ce5409289ccf4aad8d62b13a72d0009e/users/6cdd0a7f3f12414ea85ebffc9d531773/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:55.230902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.272317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b96b1e-90d8-43f4-8144-fa32d9c5fb39 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.293012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc76ade0-596d-4260-8a0d-6deb15a337f6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.298200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43af5e0e-9f5d-4af5-9b8b-fb889f460d77 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.298866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af5e0e-9f5d-4af5-9b8b-fb889f460d77 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.299176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af5e0e-9f5d-4af5-9b8b-fb889f460d77 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.299453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43af5e0e-9f5d-4af5-9b8b-fb889f460d77 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.300161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af5e0e-9f5d-4af5-9b8b-fb889f460d77 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.302098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43af5e0e-9f5d-4af5-9b8b-fb889f460d77 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.327899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.328570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.328850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.329122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.329801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.331752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.407191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 797/1583] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/users => generated 284 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.408531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.408829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.409114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.427067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:55.428326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:55.438855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 787/1584] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/users => generated 290 bytes in 227 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.438855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.446680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab1182f2-799c-4671-aa87-61b4e3a84713 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.451310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.451977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.452259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.452551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.453232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.455123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.482104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:55.499949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 798/1585] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:55.500792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.510596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a684c7d5-12b0-4a8c-887d-46a1a6571a94 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.515126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.515769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.516098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.516382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.517107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.518705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.528367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 799/1586] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:55.529117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.537859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 788/1587] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/projects => generated 371 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.537859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.540198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5811f0d-f72f-44a7-b136-1b1242128227 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.585088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:55.599543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2855a8f1-4862-4905-b595-869c73a2572a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.600323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a8f1-4862-4905-b595-869c73a2572a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.600613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a8f1-4862-4905-b595-869c73a2572a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.600891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2855a8f1-4862-4905-b595-869c73a2572a admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.601576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a8f1-4862-4905-b595-869c73a2572a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.602640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49bbdd98-201f-4f68-9232-7181b1cd77ad admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.603513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2855a8f1-4862-4905-b595-869c73a2572a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.607357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.608054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.608338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.608664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.609849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.611900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.689934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 800/1588] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/users => generated 284 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.691267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.691563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.691831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.708480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:55.709688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:55.716280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 789/1589] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/users => generated 294 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:55.717265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.717505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 801/1590] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:55.718806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.728438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-329953b3-491b-4450-ab99-a1a43e3a7dbb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.728853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e409a3f1-fd08-4923-89c7-e1ff4ce1ccf6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.738674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.738823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.738929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.738929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.739122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.739213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.739309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.739309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.739489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.739591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.739688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.739775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.747712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 802/1591] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:55.748657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.751425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 790/1592] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:55.752284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.762948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d861cd84-c3d3-4797-bd6c-fc2027bc076c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.768072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.768952 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.769238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.769517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] PATH_INFO: `/v3/projects/918001434f5c4d76a3ab9c46616eec67/users/13d8c04611e24dffad16f8bb97e111f2/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.785263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=918001434f5c4d76a3ab9c46616eec67, user_id=13d8c04611e24dffad16f8bb97e111f2, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.787971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.819682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c50f191-1461-4762-9df1-914a5bf102a8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.825117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.825949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.826232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.826508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.862010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 791/1593] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:55 2022] PUT /identity/v3/projects/918001434f5c4d76a3ab9c46616eec67/users/13d8c04611e24dffad16f8bb97e111f2/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:55.862887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.872003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:55.882865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.884521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.893911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 803/1594] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:55 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:55.894691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.904144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dda60aef-aa3d-4972-a0e3-eb995fd1f58f service nova] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.928640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d717f9a1-3317-4839-bb2f-3a7d8cf71c7c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.933388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.934027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.934314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.934585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.935270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.937306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.964510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.965324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.965708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.965996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.966730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.969386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:55.979423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 804/1595] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:55.980277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:55.990076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1119bde7-7c17-4901-b31a-3939d9eeffca admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:55.994818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:55.995495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:55.995777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:55.996083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:55.996779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:55.998314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.008361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 805/1596] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.009206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.028610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 792/1597] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:55 2022] POST /identity/v3/users => generated 286 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:56.030069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.049000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cda2c93c-40b1-4baa-88ce-075ae9a5ad92 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.053820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.055025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.055311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.055583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.056679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.058430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.068772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0777056f-ba8b-4e54-bf86-f48f8a47f190 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.072339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 793/1598] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.072339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.075402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cee042d6-e5a0-4443-a35a-3b683c4cae11 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.076079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee042d6-e5a0-4443-a35a-3b683c4cae11 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.076815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee042d6-e5a0-4443-a35a-3b683c4cae11 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.076815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cee042d6-e5a0-4443-a35a-3b683c4cae11 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.081066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75ca2a10-adac-4985-8126-52c07d2680dc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.086632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.086813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee042d6-e5a0-4443-a35a-3b683c4cae11 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.087295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.087584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.087885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.088452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cee042d6-e5a0-4443-a35a-3b683c4cae11 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.088735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.090316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.099333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 806/1599] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:56.100781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.101075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.101351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.105728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 794/1600] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.106219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.117573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ca7c-22ce-47a7-8ff4-edee445c21e4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.120438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:56.121726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.123071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.123982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.124997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.125656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.126548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.128338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.135182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 807/1601] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:56.136343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.144364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 795/1602] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.144364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.150047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a62e09d-e0fb-4502-bf85-a748314902a2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.152297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb894d01-fa06-4a67-b7b9-4d6bd4c3257d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.154720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.155446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.155929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.156220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.156960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.157086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.157686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.157971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.158249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.158552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.158970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.160591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.169017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 808/1603] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.170079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.170526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 796/1604] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.173936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.181137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49518ca4-5275-45af-a3dc-34654d95dafc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.183613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c3f361b-7a2c-4545-9c81-7df1831e698d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.185788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61d761a2-fc7b-4fbd-aaae-e64a881c7cb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.187204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d761a2-fc7b-4fbd-aaae-e64a881c7cb7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.187549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d761a2-fc7b-4fbd-aaae-e64a881c7cb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.187858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61d761a2-fc7b-4fbd-aaae-e64a881c7cb7 admin admin] PATH_INFO: `/v3/projects/7f23e9a6fbe44c69bb3e4ae47bfe93dc/users/fae075fce4d94d679a8dc5f508a1c3cf/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.188710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.189565 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.189846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.190118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] PATH_INFO: `/v3/projects/9c4b1a55929242ba9e48d4d460491fef/users/7f0caf1c16464e39b18d57ad8622e5ec/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.203269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d761a2-fc7b-4fbd-aaae-e64a881c7cb7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f23e9a6fbe44c69bb3e4ae47bfe93dc, user_id=fae075fce4d94d679a8dc5f508a1c3cf, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.205338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=9c4b1a55929242ba9e48d4d460491fef, user_id=7f0caf1c16464e39b18d57ad8622e5ec, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.207965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61d761a2-fc7b-4fbd-aaae-e64a881c7cb7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.208197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.313913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 809/1605] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:56 2022] PUT /identity/v3/projects/7f23e9a6fbe44c69bb3e4ae47bfe93dc/users/fae075fce4d94d679a8dc5f508a1c3cf/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 144 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:56.315368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2349ca-c744-4747-bae9-41ced325dcc2 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.315659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2349ca-c744-4747-bae9-41ced325dcc2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.315966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2349ca-c744-4747-bae9-41ced325dcc2 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.317448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 810/1606] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 11:55:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:55:56.318160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2349ca-c744-4747-bae9-41ced325dcc2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.319783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cc2349ca-c744-4747-bae9-41ced325dcc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.336015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 797/1607] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:56 2022] PUT /identity/v3/projects/9c4b1a55929242ba9e48d4d460491fef/users/7f0caf1c16464e39b18d57ad8622e5ec/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:56.336820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.355813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:56.357511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:56.381051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2349ca-c744-4747-bae9-41ced325dcc2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.401085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4f5f990-c972-4ffa-a7ff-8a9f5a4bd8bc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.406249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.407528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.407808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.408117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] PATH_INFO: `/v3/projects/3fa63f46e07f4f5897fa7415bf0fbb9d/users/3d2fe2b841324772955e6e95359d0870/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.424428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3fa63f46e07f4f5897fa7415bf0fbb9d, user_id=3d2fe2b841324772955e6e95359d0870, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.427144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.433311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.433983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.434281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.434545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.449843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.452475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.462389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 811/1608] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:56.463252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.473016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790b703b-2219-4db1-8cfe-4cbea06533b6 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.478161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb71fe56-d9b0-4c77-93cd-f3a2e1f6dc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.479215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71fe56-d9b0-4c77-93cd-f3a2e1f6dc85 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.482310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71fe56-d9b0-4c77-93cd-f3a2e1f6dc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.482310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb71fe56-d9b0-4c77-93cd-f3a2e1f6dc85 admin admin] PATH_INFO: `/v3/projects/659db25373614c8da6fc81e59ef9c6f6/users/97ce6d944eb74b9794d6d0d5f18d4bad/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.498283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71fe56-d9b0-4c77-93cd-f3a2e1f6dc85 admin admin] RBAC: Authorizing `identity:create_grant(project_id=659db25373614c8da6fc81e59ef9c6f6, user_id=97ce6d944eb74b9794d6d0d5f18d4bad, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.501602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb71fe56-d9b0-4c77-93cd-f3a2e1f6dc85 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.575934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 798/1609] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:56 2022] PUT /identity/v3/projects/3fa63f46e07f4f5897fa7415bf0fbb9d/users/3d2fe2b841324772955e6e95359d0870/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:56.576776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.614189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 812/1610] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:56 2022] PUT /identity/v3/projects/659db25373614c8da6fc81e59ef9c6f6/users/97ce6d944eb74b9794d6d0d5f18d4bad/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 151 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:56.616175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd35ba-123c-4050-ac86-dfca3dfd42f6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.618843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd35ba-123c-4050-ac86-dfca3dfd42f6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.618843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71bd35ba-123c-4050-ac86-dfca3dfd42f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.640037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-71bd35ba-123c-4050-ac86-dfca3dfd42f6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:56.641344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-71bd35ba-123c-4050-ac86-dfca3dfd42f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.652303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f1d741f-f2b9-43f3-873a-1d171d9114d6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.725527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 813/1611] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:56.726910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179203c-7951-458f-82a9-557d117ee6e7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.727330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179203c-7951-458f-82a9-557d117ee6e7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.727723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c179203c-7951-458f-82a9-557d117ee6e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.736633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997269c0-43ce-458a-a4b3-8e8d25387bf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.737484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997269c0-43ce-458a-a4b3-8e8d25387bf2 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.737848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997269c0-43ce-458a-a4b3-8e8d25387bf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.738198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997269c0-43ce-458a-a4b3-8e8d25387bf2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.739218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997269c0-43ce-458a-a4b3-8e8d25387bf2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.740982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997269c0-43ce-458a-a4b3-8e8d25387bf2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.751760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c179203c-7951-458f-82a9-557d117ee6e7 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:56.753151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c179203c-7951-458f-82a9-557d117ee6e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.764105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 799/1612] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/roles => generated 1420 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:56.765534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b7cc-c882-4956-9815-635cba0b6bfb None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.765865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b7cc-c882-4956-9815-635cba0b6bfb None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.766843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3449b7cc-c882-4956-9815-635cba0b6bfb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.795249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-3449b7cc-c882-4956-9815-635cba0b6bfb None None] MFA Rules not processed for user `7f0caf1c16464e39b18d57ad8622e5ec`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:56.797064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3449b7cc-c882-4956-9815-635cba0b6bfb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.821516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:56.832789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 814/1613] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:56.834117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.834411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.834827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.845839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:56.864783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] MFA Rules not processed for user `3d2fe2b841324772955e6e95359d0870`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:56.866604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.882983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 800/1614] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:56.884367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.884743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.885223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.902930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:56.904299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:56.913422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 801/1615] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:56.918270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.941986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 815/1616] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:56 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:56.948915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:56.956008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8497c03d-16e8-4582-b6b9-ab8e75910dbf None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.961937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.962939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.963299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.963655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] PATH_INFO: `/v3/projects/cbedf58e12a1470f900d7a4745774c2f/users/987801f54c03425cb457dd880531ddd5/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.979526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba572aa3-f3ec-40a9-80c1-b3d7c67141d7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:56.981182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cbedf58e12a1470f900d7a4745774c2f, user_id=987801f54c03425cb457dd880531ddd5, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.983559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:56.984855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:56.984972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:56.985288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:56.985646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:56.996637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:56.998595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.008267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 802/1617] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:57.009026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.064294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cd30f9-8d7d-476a-bf5c-4b303d1cd33f service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.067910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.068733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.069017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.069294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.098374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.098374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.098830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 816/1618] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:56 2022] PUT /identity/v3/projects/cbedf58e12a1470f900d7a4745774c2f/users/987801f54c03425cb457dd880531ddd5/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:57.098830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.102148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 803/1619] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:57.103240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.113395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c508e5dc-8d0a-4d36-9fa1-19836eb34623 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.169396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.170118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.170460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.170786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.176796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ffb8d-d8c5-4837-b170-cbd391387d06 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.181997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9697b2db-9317-464e-b96f-6a7051c2df84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.182758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.183281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b2db-9317-464e-b96f-6a7051c2df84 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.183737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b2db-9317-464e-b96f-6a7051c2df84 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.184189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9697b2db-9317-464e-b96f-6a7051c2df84 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.184320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.185168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b2db-9317-464e-b96f-6a7051c2df84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.187755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9697b2db-9317-464e-b96f-6a7051c2df84 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.196176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 804/1620] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:57.197138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.229308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:57.283301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343d5a9d-b603-4a4b-ab20-3e514ae8cca3 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.287387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.288213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.288540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.288885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.315271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 817/1621] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:57 2022] POST /identity/v3/projects => generated 367 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:57.317064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.317545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.318310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.363859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] MFA Rules not processed for user `987801f54c03425cb457dd880531ddd5`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:57.365433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:57.372425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.375507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.400687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 805/1622] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:57 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:57.401860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.432239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98c18a06-cacc-4428-ac9e-cb53f9ce6342 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.446111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:57.504601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 818/1623] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:57 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:57.509291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.513882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.514875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.515158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.515434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.516311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.518315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.531562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7df1c0-321e-45ff-b515-9e7e46c44950 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.554253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.554969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.555257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.556085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.556794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.558743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.691289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 806/1624] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:57 2022] POST /identity/v3/users => generated 290 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:57.697747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.707125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 819/1625] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:55:57 2022] POST /identity/v3/projects => generated 371 bytes in 198 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:57.708245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.711368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45085d58-7126-43bf-9fff-e6f2b9964205 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.791077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b788aea-d41e-4140-bcde-8db860a2a18a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.797580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.798445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.798670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.798804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.799265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.799687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.800792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.801259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.804533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.811062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.812496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.815931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.820177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:57.837408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 807/1626] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 140 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:55:57.837851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.862377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c87e00-5922-4cde-8a9c-922fd11b0d4a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.871466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.872680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.873028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.873312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] PATH_INFO: `/v3/projects/3b7b6d71bf114281aa7ee5f677bfa0e4/users/8ddc146f014d4d068fcd2fc8728827d0/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.890914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7b6d71bf114281aa7ee5f677bfa0e4, user_id=8ddc146f014d4d068fcd2fc8728827d0, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.894158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.923280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 820/1627] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:55:57 2022] POST /identity/v3/users => generated 292 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:55:57.924341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:57.950476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9969a47c-06b5-4be7-8a9d-cb45a82cdf46 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:57.961767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:57.962473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:57.962758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:57.963043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:57.963786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:57.965843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:57.992318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 821/1628] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:57 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:57.993692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:58.008365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 808/1629] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:57 2022] PUT /identity/v3/projects/3b7b6d71bf114281aa7ee5f677bfa0e4/users/8ddc146f014d4d068fcd2fc8728827d0/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:55:58.012805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:58.040415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa9aae0b-7dca-4bf2-8a3a-52be8be9a2a4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:58.132356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7d0b41-235e-4065-8399-96fc91a1425e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:58.140288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:58.144843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.144843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.144843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] PATH_INFO: `/v3/projects/ab8cdb0061ec49a0a3dfa8b8e0e5540b/users/98396d6c0ca3445896a17bffcb242a65/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.156165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:58.158825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.159135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.160177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.162710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab8cdb0061ec49a0a3dfa8b8e0e5540b, user_id=98396d6c0ca3445896a17bffcb242a65, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:58.163147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:58.164834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:58.166474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:58.188034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 809/1630] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:58.192210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:58.206926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65388ba5-7470-405a-9787-5d4825076684 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:58.214244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c47b9299-049d-46d8-b569-7d3dd906ebf4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:58.215802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9299-049d-46d8-b569-7d3dd906ebf4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.216531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9299-049d-46d8-b569-7d3dd906ebf4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.216870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c47b9299-049d-46d8-b569-7d3dd906ebf4 admin admin] PATH_INFO: `/v3/projects/3b7b6d71bf114281aa7ee5f677bfa0e4/users/8ddc146f014d4d068fcd2fc8728827d0/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.220428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b9299-049d-46d8-b569-7d3dd906ebf4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3b7b6d71bf114281aa7ee5f677bfa0e4, user_id=8ddc146f014d4d068fcd2fc8728827d0, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:58.223506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c47b9299-049d-46d8-b569-7d3dd906ebf4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:58.238960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 810/1631] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:58 2022] PUT /identity/v3/projects/3b7b6d71bf114281aa7ee5f677bfa0e4/users/8ddc146f014d4d068fcd2fc8728827d0/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 48 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:55:58.252813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.253641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.254907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.307582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] MFA Rules not processed for user `8ddc146f014d4d068fcd2fc8728827d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:58.308820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:58.353502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 822/1632] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:57 2022] PUT /identity/v3/projects/ab8cdb0061ec49a0a3dfa8b8e0e5540b/users/98396d6c0ca3445896a17bffcb242a65/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 360 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:58.357333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:58.401355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:58.489328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 811/1633] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:58 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 237 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:58.515046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89e86506-20f2-4102-9b65-13ecbee36ce7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:58.534152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:58.535049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.535672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.535672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.536581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:58.546626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:58.571713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 823/1634] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:55:58 2022] GET /identity/v3/roles => generated 1420 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:55:58.577287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:58.589777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:58.698918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be10dd-501c-48f0-8174-2447a0bffca0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:58.702561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6485285a-172e-46c1-89a0-636cebf85a0f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:58.704441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb60ed26-aab2-4e6b-be3e-a1d187d5ec7c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:58.705565 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60ed26-aab2-4e6b-be3e-a1d187d5ec7c admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.705937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60ed26-aab2-4e6b-be3e-a1d187d5ec7c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.706277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb60ed26-aab2-4e6b-be3e-a1d187d5ec7c admin admin] PATH_INFO: `/v3/projects/ab8cdb0061ec49a0a3dfa8b8e0e5540b/users/98396d6c0ca3445896a17bffcb242a65/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.708491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60ed26-aab2-4e6b-be3e-a1d187d5ec7c admin admin] RBAC: Authorizing `identity:create_grant(project_id=ab8cdb0061ec49a0a3dfa8b8e0e5540b, user_id=98396d6c0ca3445896a17bffcb242a65, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:58.709474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:58.711746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb60ed26-aab2-4e6b-be3e-a1d187d5ec7c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:58.712243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.712633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.713261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.743717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 812/1635] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:55:58 2022] PUT /identity/v3/projects/ab8cdb0061ec49a0a3dfa8b8e0e5540b/users/98396d6c0ca3445896a17bffcb242a65/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 157 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:55:58.745397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246099-aa2f-4cbd-bcf0-87b213e27d19 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:58.745805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246099-aa2f-4cbd-bcf0-87b213e27d19 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:58.764010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50246099-aa2f-4cbd-bcf0-87b213e27d19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:58.776710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:55:58.815007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-50246099-aa2f-4cbd-bcf0-87b213e27d19 None None] MFA Rules not processed for user `98396d6c0ca3445896a17bffcb242a65`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:58.818648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-50246099-aa2f-4cbd-bcf0-87b213e27d19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:58.846547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:58.853140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:58.885365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 824/1636] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:58 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:59.026328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 813/1637] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:55:58 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 282 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:59.042703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:59.065227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-615af912-9891-4b13-967f-e80614ff8998 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:59.107104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf33bda-122d-48a8-8481-33b21f6ff127 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:59.107399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf33bda-122d-48a8-8481-33b21f6ff127 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:59.107679 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cf33bda-122d-48a8-8481-33b21f6ff127 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:59.109564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 814/1638] 10.222.0.40 () {54 vars in 961 bytes} [Thu Jun 16 11:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:55:59.117760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e4fea-f6f8-42a5-8771-ccb8a4f9d463 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:59.118102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e4fea-f6f8-42a5-8771-ccb8a4f9d463 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:59.118340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86e4fea-f6f8-42a5-8771-ccb8a4f9d463 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:59.164839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd35abc9-1243-42f7-9edf-871b1cdd3972 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:59.166102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35abc9-1243-42f7-9edf-871b1cdd3972 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:59.166102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35abc9-1243-42f7-9edf-871b1cdd3972 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:59.166468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd35abc9-1243-42f7-9edf-871b1cdd3972 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:59.167113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c86e4fea-f6f8-42a5-8771-ccb8a4f9d463 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:59.176454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c86e4fea-f6f8-42a5-8771-ccb8a4f9d463 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:59.178730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35abc9-1243-42f7-9edf-871b1cdd3972 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:59.180223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd35abc9-1243-42f7-9edf-871b1cdd3972 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:59.193560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 825/1639] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:59 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 152 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:59.344022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 815/1640] 10.222.0.40 () {58 vars in 1058 bytes} [Thu Jun 16 11:55:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 226 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:59.348454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb070-364a-4b18-bbc3-21028ba2d554 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:59.348917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb070-364a-4b18-bbc3-21028ba2d554 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:59.349294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-557cb070-364a-4b18-bbc3-21028ba2d554 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:59.350909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 826/1641] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 11:55:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:55:59.356537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:59.358135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:59.358618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:59.393930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:55:59.395687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:55:59.413840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 816/1642] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:55:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:55:59.451852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb070-364a-4b18-bbc3-21028ba2d554 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:55:59.468416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-557cb070-364a-4b18-bbc3-21028ba2d554 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:55:59.477164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-afa2139d-1d84-44de-ad1a-7f12d8efbaad service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:55:59.477164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2139d-1d84-44de-ad1a-7f12d8efbaad service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:55:59.477164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2139d-1d84-44de-ad1a-7f12d8efbaad service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:55:59.477164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-afa2139d-1d84-44de-ad1a-7f12d8efbaad service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:55:59.494016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2139d-1d84-44de-ad1a-7f12d8efbaad service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:55:59.495489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-afa2139d-1d84-44de-ad1a-7f12d8efbaad service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:55:59.521282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 827/1643] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:55:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 70 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:55:59.704128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:01.574305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:01.696054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:01.737006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1aceb000-1dd1-4db6-81f3-67eeba088899 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:01.784799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:01.793011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:01.793307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:01.793594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] PATH_INFO: `/v3/users/987801f54c03425cb457dd880531ddd5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:01.809346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] RBAC: Authorizing `identity:delete_user(user_id=987801f54c03425cb457dd880531ddd5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:01.809346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: WARNING py.warnings [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:56:01.809346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: warnings.warn(msg) Jun 16 11:56:01.809346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: Jun 16 11:56:01.814802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:02.146131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '987801f54c03425cb457dd880531ddd5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:02.146131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '987801f54c03425cb457dd880531ddd5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:02.271785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '987801f54c03425cb457dd880531ddd5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:02.286798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '987801f54c03425cb457dd880531ddd5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:02.324630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '987801f54c03425cb457dd880531ddd5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:02.341948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 817/1644] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:01 2022] DELETE /identity/v3/users/987801f54c03425cb457dd880531ddd5 => generated 0 bytes in 765 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:02.345108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:02.345574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:02.346142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:02.378274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:02.379530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:02.535474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 828/1645] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:02.573965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:02.575601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:02.821458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e1a866-e210-48cb-a1b0-746e03cf6b30 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:02.829828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2795781-5dec-4d15-b496-02acfb1dc36e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:02.830506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2795781-5dec-4d15-b496-02acfb1dc36e service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:02.830986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2795781-5dec-4d15-b496-02acfb1dc36e service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:02.831272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2795781-5dec-4d15-b496-02acfb1dc36e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:02.882619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2795781-5dec-4d15-b496-02acfb1dc36e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:02.891098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2795781-5dec-4d15-b496-02acfb1dc36e service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:02.914015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 818/1646] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 340 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:03.090445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:03.091983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:03.114842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-853a93c4-9695-4fde-8e05-66e024f2c5ce None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:03.130241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:03.131204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:03.131491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:03.131763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] PATH_INFO: `/v3/users/fe76d1513055439987cff3d6819b2639` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:03.133433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe76d1513055439987cff3d6819b2639)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:03.135534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:03.441312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe76d1513055439987cff3d6819b2639'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:03.448814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe76d1513055439987cff3d6819b2639'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:03.588653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe76d1513055439987cff3d6819b2639'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:03.601149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe76d1513055439987cff3d6819b2639'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:03.642521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe76d1513055439987cff3d6819b2639'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:03.666830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 829/1647] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:03 2022] DELETE /identity/v3/users/fe76d1513055439987cff3d6819b2639 => generated 0 bytes in 577 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:03.680593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:03.680593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:03.680593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:03.752749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:03.760431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:04.000576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 819/1648] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 324 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:04.121618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:04.123216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:04.262514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09420b3e-c6de-4cea-bdc9-3dc1c406fc44 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:04.282983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:04.283685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:04.284001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:04.284288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:04.319754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:04.328217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:04.361796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 830/1649] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 241 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:04.961217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:04.972001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:04.980455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:04.981310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:05.025515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413d3216-a964-4671-a7bf-e2e2cf9aa4bf None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:05.029401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d15be0b-1641-4e62-93e6-7f5de9d3b540 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:05.042363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:05.045225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:05.046783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:05.046957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:05.047446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:05.048266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] PATH_INFO: `/v3/projects/cbedf58e12a1470f900d7a4745774c2f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:05.052593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:05.053696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cbedf58e12a1470f900d7a4745774c2f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:05.055451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] PATH_INFO: `/v3/users/7f0caf1c16464e39b18d57ad8622e5ec` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:05.060584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7f0caf1c16464e39b18d57ad8622e5ec)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:05.063908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:05.072299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:05.074236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:05.096153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:05.448165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cbedf58e12a1470f900d7a4745774c2f', 'request_id': 'req-4d7e6211-f2c4-4689-8a48-08d8478af715'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.454523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7f0caf1c16464e39b18d57ad8622e5ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.456271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7f0caf1c16464e39b18d57ad8622e5ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.484070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cbedf58e12a1470f900d7a4745774c2f', 'request_id': 'req-4d7e6211-f2c4-4689-8a48-08d8478af715'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.578976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7f0caf1c16464e39b18d57ad8622e5ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.589649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7f0caf1c16464e39b18d57ad8622e5ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.644252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e9d208e2-d13c-4d0a-9c5f-b989a7dc3f72 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7f0caf1c16464e39b18d57ad8622e5ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.660581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 831/1650] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:04 2022] DELETE /identity/v3/users/7f0caf1c16464e39b18d57ad8622e5ec => generated 0 bytes in 690 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:05.669221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:05.669516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:05.680923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cbedf58e12a1470f900d7a4745774c2f', 'request_id': 'req-4d7e6211-f2c4-4689-8a48-08d8478af715'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.683225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:05.700705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] The token cache is being invalidate because project cbedf58e12a1470f900d7a4745774c2f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:05.708774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.724563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:05.730245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 820/1651] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:04 2022] DELETE /identity/v3/projects/cbedf58e12a1470f900d7a4745774c2f => generated 0 bytes in 767 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:05.731719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:05.741287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:05.777402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:05.778662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:05.938757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 832/1652] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 271 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:05.978695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7e6211-f2c4-4689-8a48-08d8478af715 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:05.987968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:05.988879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:05.989170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:05.989451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] PATH_INFO: `/v3/users/98396d6c0ca3445896a17bffcb242a65` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:05.994578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] RBAC: Authorizing `identity:delete_user(user_id=98396d6c0ca3445896a17bffcb242a65)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:05.996623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:06.032243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:06.035253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:06.213991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea891fc5-223e-4608-b164-aaf536749741 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:06.219379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:06.220533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:06.221060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:06.221605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:06.236745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:06.238842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:06.254082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 833/1653] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 222 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:06.267540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '98396d6c0ca3445896a17bffcb242a65'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:06.268802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '98396d6c0ca3445896a17bffcb242a65'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:06.355748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '98396d6c0ca3445896a17bffcb242a65'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:06.368524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '98396d6c0ca3445896a17bffcb242a65'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:06.418046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-47175a7a-42ac-4e0e-a862-12c68b2bef16 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '98396d6c0ca3445896a17bffcb242a65'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:06.444925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 821/1654] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:05 2022] DELETE /identity/v3/users/98396d6c0ca3445896a17bffcb242a65 => generated 0 bytes in 712 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:06.616240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:06.618493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:06.659988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c87571f7-3534-4ab8-94d1-d2d4b3f0152f service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:06.860688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:06.861104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:06.861397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:06.861679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] PATH_INFO: `/v3/projects/e8013c8355144c5799d338a4065584f5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:06.863812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e8013c8355144c5799d338a4065584f5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:06.865183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: WARNING py.warnings [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jun 16 11:56:06.865183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: warnings.warn(msg) Jun 16 11:56:06.865183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: Jun 16 11:56:06.868038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:06.987511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e8013c8355144c5799d338a4065584f5', 'request_id': 'req-acb2cc46-4289-4288-9d57-1820ede97ed8'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:07.006222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e8013c8355144c5799d338a4065584f5', 'request_id': 'req-acb2cc46-4289-4288-9d57-1820ede97ed8'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:07.014849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e8013c8355144c5799d338a4065584f5', 'request_id': 'req-acb2cc46-4289-4288-9d57-1820ede97ed8'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:07.109364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] The token cache is being invalidate because project e8013c8355144c5799d338a4065584f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:07.109999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:07.112439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-acb2cc46-4289-4288-9d57-1820ede97ed8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:07.114483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 834/1655] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:06 2022] DELETE /identity/v3/projects/e8013c8355144c5799d338a4065584f5 => generated 0 bytes in 499 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:07.119800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:07.120280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:07.120572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:07.131249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:07.156924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:07.158339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:07.173867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:07.316394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 822/1656] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 198 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:07.324218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:07.324583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:07.324957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:07.356639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:07.358061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:07.388589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 835/1657] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:07.395487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:07.416222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9bbf67b-6da7-4306-89b3-5bd416030260 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:07.433441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:07.435167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:07.435465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:07.435747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:07.438741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:07.439123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:07.442846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:07.471124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 823/1658] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:07 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:07.473234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:07.488411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d248e895-579a-4cd3-bf1d-d2cc69b5f368 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:07.493508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:07.494255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:07.494545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:07.494871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:07.495624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:07.501711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:07.673638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 836/1659] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:07 2022] POST /identity/v3/projects => generated 403 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:07.677987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:07.708738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2ecce0d-7767-49ee-a23a-c965e82880e1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:07.844559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:07.845842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:07.846137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:07.846489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:07.847194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:07.849427 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:07.979923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 824/1660] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:07 2022] POST /identity/v3/users => generated 302 bytes in 300 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:07.981287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:08.001647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c904ced3-342d-471c-9f4c-7836b0b09e2a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:08.017676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:08.018401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.020034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.020503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.023190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:08.029685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:08.064541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 837/1661] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:07 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:08.064700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:08.078632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc582bef-130b-41a4-b152-63ee7a5bbf55 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:08.092164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:08.093993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.094447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.094864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.096174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:08.098551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:08.120597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 825/1662] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:08 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:08.125769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:08.138708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bf7986e-53d5-4804-82c9-b506a5071591 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:08.146828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:08.148428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.148922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.149481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] PATH_INFO: `/v3/projects/3d978da939764925bca041e365a1cf5f/users/72b04b1ecc214237900350ae7867daab/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.189295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d978da939764925bca041e365a1cf5f, user_id=72b04b1ecc214237900350ae7867daab, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:08.193544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:08.232488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:08.234767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:08.284912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f7c2cc-6d71-4455-a36f-488d34260bbc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:08.302570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:08.304146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.304780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.305150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] PATH_INFO: `/v3/projects/ab8cdb0061ec49a0a3dfa8b8e0e5540b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.309275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ab8cdb0061ec49a0a3dfa8b8e0e5540b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:08.321982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 838/1663] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:08 2022] PUT /identity/v3/projects/3d978da939764925bca041e365a1cf5f/users/72b04b1ecc214237900350ae7867daab/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:08.323543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:08.326173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:08.328519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:08.525764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-009a5b46-adab-40d8-9e62-953e6172c6fa admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:08.576622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ab8cdb0061ec49a0a3dfa8b8e0e5540b', 'request_id': 'req-c259635e-f33a-41db-a9c1-15401135bd87'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.615401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ab8cdb0061ec49a0a3dfa8b8e0e5540b', 'request_id': 'req-c259635e-f33a-41db-a9c1-15401135bd87'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.690189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:08.691289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.691760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.692053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] PATH_INFO: `/v3/projects/9c4b1a55929242ba9e48d4d460491fef` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.693402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9c4b1a55929242ba9e48d4d460491fef)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:08.695545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:08.704019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ab8cdb0061ec49a0a3dfa8b8e0e5540b', 'request_id': 'req-c259635e-f33a-41db-a9c1-15401135bd87'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.735241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] The token cache is being invalidate because project ab8cdb0061ec49a0a3dfa8b8e0e5540b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:08.736276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.738175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c259635e-f33a-41db-a9c1-15401135bd87 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.740697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 826/1664] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:08 2022] DELETE /identity/v3/projects/ab8cdb0061ec49a0a3dfa8b8e0e5540b => generated 0 bytes in 509 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:08.742874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6af29e-2870-48ac-92f3-fe0167d56278 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.743369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6af29e-2870-48ac-92f3-fe0167d56278 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.743853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd6af29e-2870-48ac-92f3-fe0167d56278 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.816573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-dd6af29e-2870-48ac-92f3-fe0167d56278 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:08.819687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-dd6af29e-2870-48ac-92f3-fe0167d56278 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:08.839716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 827/1665] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:08.841648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:08.842124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:08.852063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:08.862301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9c4b1a55929242ba9e48d4d460491fef', 'request_id': 'req-2b171bf1-2e0d-4271-9cdf-766da2dad078'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.864063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:08.865473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:08.874631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9c4b1a55929242ba9e48d4d460491fef', 'request_id': 'req-2b171bf1-2e0d-4271-9cdf-766da2dad078'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.882406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9c4b1a55929242ba9e48d4d460491fef', 'request_id': 'req-2b171bf1-2e0d-4271-9cdf-766da2dad078'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:08.942377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:08.952579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 828/1666] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:08.954331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:08.957114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:09.011631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] The token cache is being invalidate because project 9c4b1a55929242ba9e48d4d460491fef was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:09.012022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.013015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2b171bf1-2e0d-4271-9cdf-766da2dad078 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.016548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 839/1667] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:08 2022] DELETE /identity/v3/projects/9c4b1a55929242ba9e48d4d460491fef => generated 0 bytes in 693 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:09.017960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.018308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.018602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.039676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e756a38d-814b-4ea2-b860-4b94933f80e5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.051561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.052634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.053343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.053814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.056084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:09.074984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:09.078457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:09.082380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.084429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.097545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 829/1668] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:09.098603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.099679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 840/1669] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:09.100623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:09.107637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.114433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61993f2-6f93-4263-9958-7467190cce6d service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.119711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.120943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.122111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.122624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] PATH_INFO: `/v3/users/8ddc146f014d4d068fcd2fc8728827d0` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.124373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] RBAC: Authorizing `identity:delete_user(user_id=8ddc146f014d4d068fcd2fc8728827d0)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.126762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.138732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f723f40-c395-4edc-9bca-5f0d0f94f5ff None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.153916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.154619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.155829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.156318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.157214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:09.157671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.161623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.173271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:INSERT=4,keystone:DELETE=7 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:09.180665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 841/1670] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:09.184422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.197259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca426c53-ed02-4650-ab53-53b8bf5b6368 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.204933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.205860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.206236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.207433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.208403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.213483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.337085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 842/1671] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:09 2022] POST /identity/v3/projects => generated 357 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:09.345249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.392399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8ddc146f014d4d068fcd2fc8728827d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.393596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8ddc146f014d4d068fcd2fc8728827d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.485648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4ac6f45-4f20-47c4-ae38-7da221fedc93 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.494857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.495570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.499230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.499230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.499230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.499710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.519689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8ddc146f014d4d068fcd2fc8728827d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.554954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8ddc146f014d4d068fcd2fc8728827d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.582193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8ddc146f014d4d068fcd2fc8728827d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:09.595573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 830/1672] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:09 2022] DELETE /identity/v3/users/8ddc146f014d4d068fcd2fc8728827d0 => generated 0 bytes in 497 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:09.615204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 843/1673] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:09 2022] POST /identity/v3/users => generated 279 bytes in 270 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:09.618399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.620857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:09.822208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ea5732c-e850-4bf4-ac95-6b7ea6073a1a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.841257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.843651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.843986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.850362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.851278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.856729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.888676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 831/1674] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 268 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:09.890012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.905693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d6ee54f-21cc-49bc-bcb3-bc6907b5aaaa admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.918536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb590d06-0176-4c48-8b72-9ce8d79295a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.919338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb590d06-0176-4c48-8b72-9ce8d79295a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.919704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb590d06-0176-4c48-8b72-9ce8d79295a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.920232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb590d06-0176-4c48-8b72-9ce8d79295a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:09.921735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb590d06-0176-4c48-8b72-9ce8d79295a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:09.923511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb590d06-0176-4c48-8b72-9ce8d79295a3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:09.957680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 844/1675] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:09 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:09.958700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:09.974590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31497fbf-70a2-4319-82aa-903c3cc376bd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:09.986235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:09.987243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:09.987444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:09.987719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] PATH_INFO: `/v3/projects/9cb701214ced4b65b02d69435f9f3c27/users/7274bea2ee0f4dd0a98580084c6cec2f/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:10.026027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9cb701214ced4b65b02d69435f9f3c27, user_id=7274bea2ee0f4dd0a98580084c6cec2f, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:10.028740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:10.175189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 832/1676] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:09 2022] PUT /identity/v3/projects/9cb701214ced4b65b02d69435f9f3c27/users/7274bea2ee0f4dd0a98580084c6cec2f/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:10.185446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:10.186842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:10.187305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:10.237703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] MFA Rules not processed for user `7274bea2ee0f4dd0a98580084c6cec2f`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:10.242088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:10.438726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 845/1677] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:10.443880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:10.582001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6e91266-90d1-43ea-ac64-ed00862274dd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:10.590749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250101da-1f78-49b3-b36c-9ce9e45e43ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:10.591643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250101da-1f78-49b3-b36c-9ce9e45e43ee admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:10.592771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250101da-1f78-49b3-b36c-9ce9e45e43ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:10.593856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250101da-1f78-49b3-b36c-9ce9e45e43ee admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:10.594700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250101da-1f78-49b3-b36c-9ce9e45e43ee admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:10.598778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250101da-1f78-49b3-b36c-9ce9e45e43ee admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:10.750553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 833/1678] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:10 2022] POST /identity/v3/projects => generated 357 bytes in 304 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:10.752746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:10.766935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfc861e-6330-446e-89a5-eef8eab296d9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:10.880966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:10.903396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:10.904144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:10.904444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:10.904716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:10.911786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:10.919426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:10.950399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082f2fe-54bc-4bb9-b817-f05b94b938b6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:10.950520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082f2fe-54bc-4bb9-b817-f05b94b938b6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:10.950784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6082f2fe-54bc-4bb9-b817-f05b94b938b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.005349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6082f2fe-54bc-4bb9-b817-f05b94b938b6 None None] MFA Rules not processed for user `fae075fce4d94d679a8dc5f508a1c3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:11.010571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6082f2fe-54bc-4bb9-b817-f05b94b938b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:11.041178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 846/1679] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:10 2022] POST /identity/v3/users => generated 285 bytes in 289 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:11.045097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.062664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:11.064303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f08ef5d-28ed-4bbc-a430-7b177f4150c6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:11.077983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:11.078919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.079344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.080016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.084164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:11.087685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:11.115728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 847/1680] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:11.116980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.141639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dea7404e-4315-4224-8d56-3973d2dc6581 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:11.160006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:11.161027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.161316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.161910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] PATH_INFO: `/v3/projects/2d031ae91449428aa7b26b44cc190745/users/6a4f77f9526349d1a650ba40141e6baf/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.170067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 834/1681] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:10 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:11.179564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.180227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.180355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.201154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d031ae91449428aa7b26b44cc190745, user_id=6a4f77f9526349d1a650ba40141e6baf, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:11.204784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:11.229522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] MFA Rules not processed for user `fae075fce4d94d679a8dc5f508a1c3cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:11.231050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:11.247665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 835/1682] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:11 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 69 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:11.254035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.255819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:11.339342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 848/1683] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:11 2022] PUT /identity/v3/projects/2d031ae91449428aa7b26b44cc190745/users/6a4f77f9526349d1a650ba40141e6baf/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:11.343088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.430423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-097b33ed-25fc-4cf5-9609-28db8f778723 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:11.513244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d734a9-709d-4bb3-8ea1-75f4a722624b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:11.536424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:11.537156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.537445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.540157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.540157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:11.540157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:11.571362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:11.571863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.572304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.573185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.574649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 849/1684] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:11 2022] GET /identity/v3/roles => generated 1420 bytes in 232 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:11.575781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.582804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:11.620100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5348354e-aafb-4bb1-b7ed-e519ce506c2f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:11.634861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:11.636708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.637190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.637612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] PATH_INFO: `/v3/projects/3b7b6d71bf114281aa7ee5f677bfa0e4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.640917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3b7b6d71bf114281aa7ee5f677bfa0e4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:11.643306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:11.708641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:11.710354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:11.737457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 836/1685] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:11 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 484 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:11.752208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.757394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:11.792356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc7dccf7-f417-45e3-98b2-aaf96317c0da service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:11.800592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3b7b6d71bf114281aa7ee5f677bfa0e4', 'request_id': 'req-d25d9bde-996e-4ae0-b9f7-366dc1049447'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:11.815138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3b7b6d71bf114281aa7ee5f677bfa0e4', 'request_id': 'req-d25d9bde-996e-4ae0-b9f7-366dc1049447'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:11.849083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3b7b6d71bf114281aa7ee5f677bfa0e4', 'request_id': 'req-d25d9bde-996e-4ae0-b9f7-366dc1049447'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:11.923799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:11.924758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:11.925054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:11.925333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] PATH_INFO: `/v3/users/3d2fe2b841324772955e6e95359d0870` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:11.926610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d2fe2b841324772955e6e95359d0870)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:11.928805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:11.954902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] The token cache is being invalidate because project 3b7b6d71bf114281aa7ee5f677bfa0e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:11.957678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:11.959269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:11.967640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 850/1686] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:11 2022] DELETE /identity/v3/projects/3b7b6d71bf114281aa7ee5f677bfa0e4 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:11.968101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:11.970359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:11.992530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d25d9bde-996e-4ae0-b9f7-366dc1049447 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:12.000673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c415a0e-af2d-461d-bd7c-6645b98a224d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:12.001845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c415a0e-af2d-461d-bd7c-6645b98a224d admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.002253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c415a0e-af2d-461d-bd7c-6645b98a224d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.002631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c415a0e-af2d-461d-bd7c-6645b98a224d admin admin] PATH_INFO: `/v3/projects/2d031ae91449428aa7b26b44cc190745/users/6a4f77f9526349d1a650ba40141e6baf/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.005783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c415a0e-af2d-461d-bd7c-6645b98a224d admin admin] RBAC: Authorizing `identity:create_grant(project_id=2d031ae91449428aa7b26b44cc190745, user_id=6a4f77f9526349d1a650ba40141e6baf, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:12.009291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c415a0e-af2d-461d-bd7c-6645b98a224d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:12.030062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 851/1687] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:11 2022] PUT /identity/v3/projects/2d031ae91449428aa7b26b44cc190745/users/6a4f77f9526349d1a650ba40141e6baf/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:12.032120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6011545-2e45-45c4-aff8-c0ecda3ed9cd None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.033151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6011545-2e45-45c4-aff8-c0ecda3ed9cd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.033151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6011545-2e45-45c4-aff8-c0ecda3ed9cd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.081380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e6011545-2e45-45c4-aff8-c0ecda3ed9cd None None] MFA Rules not processed for user `13d8c04611e24dffad16f8bb97e111f2`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.084360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e6011545-2e45-45c4-aff8-c0ecda3ed9cd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.193592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d2fe2b841324772955e6e95359d0870'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:12.198405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d2fe2b841324772955e6e95359d0870'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:12.230893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 852/1688] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.240187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a417d9-e6ac-4a97-8190-af45d3ea492e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.240187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a417d9-e6ac-4a97-8190-af45d3ea492e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.240187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54a417d9-e6ac-4a97-8190-af45d3ea492e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.287468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-54a417d9-e6ac-4a97-8190-af45d3ea492e None None] MFA Rules not processed for user `6cdd0a7f3f12414ea85ebffc9d531773`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.289452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-54a417d9-e6ac-4a97-8190-af45d3ea492e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.290282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d2fe2b841324772955e6e95359d0870'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:12.300815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d2fe2b841324772955e6e95359d0870'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:12.318738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5cffc007-3842-4e1d-bd34-f3c685a06f59 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d2fe2b841324772955e6e95359d0870'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:12.327226 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 837/1689] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:11 2022] DELETE /identity/v3/users/3d2fe2b841324772955e6e95359d0870 => generated 0 bytes in 578 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:12.329064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661edde7-e4d4-473b-bdf2-95674880b645 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.329542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661edde7-e4d4-473b-bdf2-95674880b645 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.330188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661edde7-e4d4-473b-bdf2-95674880b645 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.354823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-661edde7-e4d4-473b-bdf2-95674880b645 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.357008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-661edde7-e4d4-473b-bdf2-95674880b645 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.455050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 853/1690] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 224 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.473114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.481525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.481525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.486600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 838/1691] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.489214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edffdb7-34f4-45e5-810b-4732c7d5e41f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.489654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edffdb7-34f4-45e5-810b-4732c7d5e41f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.490115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8edffdb7-34f4-45e5-810b-4732c7d5e41f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.551586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] MFA Rules not processed for user `6a4f77f9526349d1a650ba40141e6baf`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.555850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.566718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8edffdb7-34f4-45e5-810b-4732c7d5e41f None None] MFA Rules not processed for user `72b04b1ecc214237900350ae7867daab`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.568776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8edffdb7-34f4-45e5-810b-4732c7d5e41f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.738508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 854/1692] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3291 bytes in 279 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.738508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:12.744640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.753787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 839/1693] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 266 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.757121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eeda9-b27d-4478-ac2d-a84d59fc6bf0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.758882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eeda9-b27d-4478-ac2d-a84d59fc6bf0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.762127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098eeda9-b27d-4478-ac2d-a84d59fc6bf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.802648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-098eeda9-b27d-4478-ac2d-a84d59fc6bf0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.807468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-098eeda9-b27d-4478-ac2d-a84d59fc6bf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.822198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 840/1694] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 68 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.824217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2307e5-abd3-4e57-92cd-9b724d82c26d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.824717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2307e5-abd3-4e57-92cd-9b724d82c26d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.832860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2307e5-abd3-4e57-92cd-9b724d82c26d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.837152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:12.866301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-3c2307e5-abd3-4e57-92cd-9b724d82c26d None None] MFA Rules not processed for user `6cdd0a7f3f12414ea85ebffc9d531773`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:12.868500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3c2307e5-abd3-4e57-92cd-9b724d82c26d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.893748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef6f9d24-e4c6-406a-84d7-c032c43397ec None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:12.918791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:12.919754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.920925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.921229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:12.925105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:12.993842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 841/1695] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:12.995327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:12.995634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:12.995942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.002516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:13.024576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.027493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-216bddae-aac4-4179-9b7b-dcbc4aabd7ee service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.035387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:13.036680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:13.041058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 855/1696] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:13.042864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b635c-2d04-45ee-97b8-5b1349953ad9 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.043164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b635c-2d04-45ee-97b8-5b1349953ad9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.043450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0b635c-2d04-45ee-97b8-5b1349953ad9 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.045158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 856/1697] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 11:56:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:13.046193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b635c-2d04-45ee-97b8-5b1349953ad9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.048508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bb0b635c-2d04-45ee-97b8-5b1349953ad9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:13.050876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 842/1698] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:13.051699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.057216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:13.086579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21cf2d90-17cc-474b-a663-cea9f6371798 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.094598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.096707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.096707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.096707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.097588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.099943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.146882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0b635c-2d04-45ee-97b8-5b1349953ad9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.154658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa47f225-fdec-4abe-8a7c-dbe4f85defe9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.156225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f225-fdec-4abe-8a7c-dbe4f85defe9 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.156505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f225-fdec-4abe-8a7c-dbe4f85defe9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.156793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa47f225-fdec-4abe-8a7c-dbe4f85defe9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.168748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f225-fdec-4abe-8a7c-dbe4f85defe9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.173273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa47f225-fdec-4abe-8a7c-dbe4f85defe9 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.187960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 843/1699] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:13 2022] POST /identity/v3/projects => generated 405 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:13.188810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.189475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 857/1700] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3291 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:13.191050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.191282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.191565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.194686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:13.247575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] MFA Rules not processed for user `516b221cb1794430befadcde5f9fd77b`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:13.249374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:13.337914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c291026-7503-4380-88cf-ea5d1242c1c6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.352161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 858/1701] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:56:13 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:13.353190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.354994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.355840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.356611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.356898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.475620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-def66cb3-85ea-40b5-bdf7-fd44f88d983c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.489437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.491013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.499503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.500299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.500592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.500877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.504472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 844/1702] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:13.505414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.506213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:13.506550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.511797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.517395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd4329aa-4db1-48a8-86e1-8deb8fd15ac9 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.523240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.527206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.527206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.527206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.527206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.528281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.565935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 859/1703] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:13.570723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.655527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 845/1704] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:13 2022] POST /identity/v3/users => generated 309 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:13.659302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.681449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9e38f7b-e211-4aff-bcfe-4713a3c15c01 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.690208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.690993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.691355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.691705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.715611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.717155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.736262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-689f903e-edb3-49da-81ca-2d825cad1737 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.742579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.743358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.743654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.743975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.752478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 846/1705] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:13.753432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.830689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.833544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.844687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 860/1706] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 274 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:13.846183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:13.859912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ee7d244-4846-475d-be53-51271529c325 service nova] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.862795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.863481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.863771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.864082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.864780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:13.866749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:13.896180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52fe8b28-3c6e-47d1-9da3-cd042b4b5b0a service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:13.904953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:13.905645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:13.905941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:13.906229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:13.910869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:14.005249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 861/1707] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:13 2022] POST /identity/v3/projects => generated 385 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:14.009777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.064591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.066155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4a52366-61f0-4723-a6f8-1ff01a6ea109 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.084682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 847/1708] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:13 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:14.095192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.096175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.096540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.182945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] MFA Rules not processed for user `97ce6d944eb74b9794d6d0d5f18d4bad`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:14.187803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:14.200117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:14.213323 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0aa8abfe-0c27-410b-9fc8-37380b6beba1 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.229583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.234528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.234528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.234528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.234528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.234528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.262731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 862/1709] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 254 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:14.264616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.290801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-414368ee-32db-4db3-a387-32caea34690a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.305212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.305916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.306295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.306630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.307411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.310208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.318838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:14.341536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 848/1710] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:14 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:14.342733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.371346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d5790d4-bffc-46ab-a8ea-8e6fb46c9b28 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.377395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.378579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.379011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.379435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] PATH_INFO: `/v3/projects/e1bc6c309ff440dba31625b6083a6b60/users/fe46d7bd88c449fda863c0a82a2ed8cf/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.398070 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bc6c309ff440dba31625b6083a6b60, user_id=fe46d7bd88c449fda863c0a82a2ed8cf, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.401389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.412218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 863/1711] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:14 2022] POST /identity/v3/users => generated 293 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:14.416567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.495440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9e0fa4e-bb2b-4629-8b30-f8b676edd452 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.504093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 849/1712] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:14 2022] PUT /identity/v3/projects/e1bc6c309ff440dba31625b6083a6b60/users/fe46d7bd88c449fda863c0a82a2ed8cf/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:14.508341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.668603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25879c27-4405-4e1c-84b8-9b625c7b239f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.691778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.692192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.692496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.692758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.693501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.695187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.697314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.698092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.698298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.706485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.706485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.707255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.725853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 864/1713] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 310 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:14.729409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.762406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 850/1714] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:14.763368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.778756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4652d85e-2332-4371-9fd0-6253788acef0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.785137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.786480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.786970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.787493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.788472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.790880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.795158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d728c4f3-5a5b-4260-af19-c97ec5615f8a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.812708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59f99542-deca-448e-8a17-c43a21fb0347 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.813636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f99542-deca-448e-8a17-c43a21fb0347 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.813925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f99542-deca-448e-8a17-c43a21fb0347 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.814208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59f99542-deca-448e-8a17-c43a21fb0347 admin admin] PATH_INFO: `/v3/projects/e1bc6c309ff440dba31625b6083a6b60/users/fe46d7bd88c449fda863c0a82a2ed8cf/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.828010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f99542-deca-448e-8a17-c43a21fb0347 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1bc6c309ff440dba31625b6083a6b60, user_id=fe46d7bd88c449fda863c0a82a2ed8cf, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:14.829069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 865/1715] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:14 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:14.830098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.830595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59f99542-deca-448e-8a17-c43a21fb0347 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:14.865746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-503b0064-7264-4719-af02-c416db61cadb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:14.886924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 851/1716] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:14 2022] PUT /identity/v3/projects/e1bc6c309ff440dba31625b6083a6b60/users/fe46d7bd88c449fda863c0a82a2ed8cf/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:14.888318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9b8d-9d71-48d0-8faa-4a365598e743 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.888614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9b8d-9d71-48d0-8faa-4a365598e743 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.888897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7eba9b8d-9d71-48d0-8faa-4a365598e743 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.896222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 852/1717] 10.222.0.40 () {54 vars in 912 bytes} [Thu Jun 16 11:56:14 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:14.897178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9b8d-9d71-48d0-8faa-4a365598e743 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:14.906135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7eba9b8d-9d71-48d0-8faa-4a365598e743 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:14.984535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-533bd474-4882-4b04-9e43-b1c48c28e90e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:14.985761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd474-4882-4b04-9e43-b1c48c28e90e admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:14.986292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd474-4882-4b04-9e43-b1c48c28e90e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:14.986766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-533bd474-4882-4b04-9e43-b1c48c28e90e admin admin] PATH_INFO: `/v3/projects/54db129703ce4fafb33196fda2ff3866/users/19ebdf8423054a94b71d12990fe56445/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:14.986867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:15.152511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7eba9b8d-9d71-48d0-8faa-4a365598e743 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:15.166225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943dd595-d8a5-48ba-80db-1df38603499d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:15.167419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd595-d8a5-48ba-80db-1df38603499d service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.167726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd595-d8a5-48ba-80db-1df38603499d service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.168035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943dd595-d8a5-48ba-80db-1df38603499d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.280726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd474-4882-4b04-9e43-b1c48c28e90e admin admin] RBAC: Authorizing `identity:create_grant(project_id=54db129703ce4fafb33196fda2ff3866, user_id=19ebdf8423054a94b71d12990fe56445, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:15.283941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-533bd474-4882-4b04-9e43-b1c48c28e90e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:15.286213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dd595-d8a5-48ba-80db-1df38603499d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:15.296288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-943dd595-d8a5-48ba-80db-1df38603499d service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:15.327042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 853/1718] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:14 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 430 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:15.329509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41eeb-6404-4ea4-9c22-b02883987796 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.330026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41eeb-6404-4ea4-9c22-b02883987796 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.330743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dc41eeb-6404-4ea4-9c22-b02883987796 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.372581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 866/1719] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:14 2022] PUT /identity/v3/projects/54db129703ce4fafb33196fda2ff3866/users/19ebdf8423054a94b71d12990fe56445/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 533 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:15.374814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f550928-f3b5-4726-8dc2-a7f76b658171 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.375212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f550928-f3b5-4726-8dc2-a7f76b658171 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.375636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f550928-f3b5-4726-8dc2-a7f76b658171 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.378311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1dc41eeb-6404-4ea4-9c22-b02883987796 None None] MFA Rules not processed for user `516b221cb1794430befadcde5f9fd77b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:15.379903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 867/1720] 10.222.0.40 () {54 vars in 960 bytes} [Thu Jun 16 11:56:15 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:15.380967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f550928-f3b5-4726-8dc2-a7f76b658171 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:15.381439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1dc41eeb-6404-4ea4-9c22-b02883987796 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:15.388742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5f550928-f3b5-4726-8dc2-a7f76b658171 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:15.517267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 854/1721] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:56:15 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:15.519334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734f6b1-42b1-40ab-8a13-c467c7514eee None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.519808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734f6b1-42b1-40ab-8a13-c467c7514eee None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.520125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2734f6b1-42b1-40ab-8a13-c467c7514eee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.531171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f550928-f3b5-4726-8dc2-a7f76b658171 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:15.536922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:15.537929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.538313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.539144 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] PATH_INFO: `/v3/projects/3fa63f46e07f4f5897fa7415bf0fbb9d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.546698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3fa63f46e07f4f5897fa7415bf0fbb9d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:15.549070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:15.626037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2734f6b1-42b1-40ab-8a13-c467c7514eee None None] MFA Rules not processed for user `19ebdf8423054a94b71d12990fe56445`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:15.627464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2734f6b1-42b1-40ab-8a13-c467c7514eee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:15.738583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3fa63f46e07f4f5897fa7415bf0fbb9d', 'request_id': 'req-cb5abf59-c4d1-4362-8f55-87890f85c9e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:15.746008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3fa63f46e07f4f5897fa7415bf0fbb9d', 'request_id': 'req-cb5abf59-c4d1-4362-8f55-87890f85c9e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:15.753179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3fa63f46e07f4f5897fa7415bf0fbb9d', 'request_id': 'req-cb5abf59-c4d1-4362-8f55-87890f85c9e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:15.785416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 855/1722] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:15 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 264 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:15.785416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.785416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.785416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.822037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:15.823096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] The token cache is being invalidate because project 3fa63f46e07f4f5897fa7415bf0fbb9d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:15.823262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:15.823583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:15.824715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:15.828482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 868/1723] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:15 2022] DELETE /identity/v3/projects/3fa63f46e07f4f5897fa7415bf0fbb9d => generated 0 bytes in 447 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:15.828482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:15.832239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:15.964131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb5abf59-c4d1-4362-8f55-87890f85c9e3 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:15.976246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 856/1724] 10.222.0.40 () {58 vars in 1057 bytes} [Thu Jun 16 11:56:15 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:15.977108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:15.977790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:15.978607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:15.978607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:15.980018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:15.982131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:15.985659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:16.001149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.005072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5b8cac7-98c8-4f4f-b648-63e4a25a42ac service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.023674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 869/1725] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:15 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:16.046818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.049254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.050995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.134661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:16.136316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:16.141181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82825fc0-8a8f-4ce1-b22f-8ec4cbe93ee0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.152363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 870/1726] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:16.153826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.199034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.199912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.200206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.202079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.204133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.212215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeec318f-5163-466d-a89b-477fde0bbebb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.212869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.231330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.233542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.233836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.236693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:16.237478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.241555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:16.267826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.269454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aba624a2-e4dd-4ba8-b9cc-4f94f147565b service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.301341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 871/1727] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 148 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:16.302920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.303210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.303487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.362072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 857/1728] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:15 2022] POST /identity/v3/projects => generated 385 bytes in 385 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:16.366280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.366735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:16.368052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:16.502442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 872/1729] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:16.509119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.529513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e0efae-7532-45a6-889a-b216c2e1374d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.545822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.546591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.546957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.547310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.548165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:16.548551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.548991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae7e3e80-651e-494d-8e89-ed3cc5204307 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.552970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.555694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.555988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.556343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.556690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.557464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.560052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.590693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 873/1730] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:16.592474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.604776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7e62ba0-e511-43d8-9340-0ebfd111d4dc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.611653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.612916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.613557 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.613909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.614855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.617104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.640687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 858/1731] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:16 2022] POST /identity/v3/users => generated 299 bytes in 275 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:16.642384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.661684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1258b5c5-3a09-477c-b2d9-a92e3848fdcc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.671258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.671577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.671860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.672267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.678378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.680162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.766077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 874/1732] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:16 2022] POST /identity/v3/projects => generated 403 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:16.769051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 859/1733] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:16 2022] GET /identity/v3/roles => generated 1420 bytes in 127 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:16.771237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.776575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:16.793088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6898b9ff-75c9-4700-9e41-03cd2c7d4670 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.861976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:16.925167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:16.925881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:16.926169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:16.926451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:16.927221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:16.933411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:16.990932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25671d1a-bd4e-4db2-b244-5a12128b0b16 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:16.997591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:17.001298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.001799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.002230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] PATH_INFO: `/v3/projects/1d21bc5503ba476caaff6898434c5c52/users/f95021839c22479abd5995687aa7992c/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:17.048386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d21bc5503ba476caaff6898434c5c52, user_id=f95021839c22479abd5995687aa7992c, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:17.051585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:17.088928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 860/1734] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:16 2022] POST /identity/v3/users => generated 302 bytes in 318 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:17.089890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:17.113547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c2bcc87-38f1-454f-955d-f78dbe96dc96 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:17.119903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:17.126483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:17.127180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.127470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.127857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:17.133468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:17.135054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:17.157908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 861/1735] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 68 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:17.165646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:17.178428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2530afae-002a-414d-8ccd-11a17edcd437 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:17.189607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 875/1736] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:16 2022] PUT /identity/v3/projects/1d21bc5503ba476caaff6898434c5c52/users/f95021839c22479abd5995687aa7992c/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:17.194197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:17.325094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:17.326040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.326508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.326932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:17.327817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:17.329624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:17.369739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 862/1737] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:17.373983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:17.393602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0bfd1e-d501-4520-abda-7662e9708dc1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:17.420029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e6ebc9b-c917-4dbb-b64e-f2475bcc0e29 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:17.428894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ebc9b-c917-4dbb-b64e-f2475bcc0e29 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.429428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ebc9b-c917-4dbb-b64e-f2475bcc0e29 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.429884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e6ebc9b-c917-4dbb-b64e-f2475bcc0e29 admin admin] PATH_INFO: `/v3/projects/ed2f14ef34524242a67069f619fad5c2/users/9ad22a72fbb345309f5f276babd350d3/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:17.474964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ebc9b-c917-4dbb-b64e-f2475bcc0e29 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed2f14ef34524242a67069f619fad5c2, user_id=9ad22a72fbb345309f5f276babd350d3, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:17.484064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e6ebc9b-c917-4dbb-b64e-f2475bcc0e29 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:17.503913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:17.544535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8b32f3-2705-4a5b-b999-37be7b09b7c0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:17.554974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:17.556999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.557294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.557579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:17.558336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:17.576192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 863/1738] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:17 2022] PUT /identity/v3/projects/ed2f14ef34524242a67069f619fad5c2/users/9ad22a72fbb345309f5f276babd350d3/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 203 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:17.580572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:17.589374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440913d5-da61-413c-8383-35a340ebca64 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.589687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440913d5-da61-413c-8383-35a340ebca64 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.589987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-440913d5-da61-413c-8383-35a340ebca64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:17.646959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 876/1739] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:17 2022] GET /identity/v3/roles => generated 1420 bytes in 453 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:17.649519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:17.685524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-709576de-b54b-48ef-95b6-2b105201ece3 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:17.694215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-440913d5-da61-413c-8383-35a340ebca64 None None] MFA Rules not processed for user `9ad22a72fbb345309f5f276babd350d3`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:17.698755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-440913d5-da61-413c-8383-35a340ebca64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:17.864554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 864/1740] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:17 2022] POST /identity/v3/auth/tokens => generated 3270 bytes in 277 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:17.872269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440913d5-da61-413c-8383-35a340ebca64 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:17.974977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ab487cb-c653-4546-b2eb-94cd87f6853f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:17.980312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab487cb-c653-4546-b2eb-94cd87f6853f admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:17.980690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab487cb-c653-4546-b2eb-94cd87f6853f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:17.982190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ab487cb-c653-4546-b2eb-94cd87f6853f admin admin] PATH_INFO: `/v3/projects/1d21bc5503ba476caaff6898434c5c52/users/f95021839c22479abd5995687aa7992c/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.000482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab487cb-c653-4546-b2eb-94cd87f6853f admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d21bc5503ba476caaff6898434c5c52, user_id=f95021839c22479abd5995687aa7992c, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:18.041763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ab487cb-c653-4546-b2eb-94cd87f6853f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:18.089956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-440913d5-da61-413c-8383-35a340ebca64 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:18.097099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 877/1741] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:17 2022] PUT /identity/v3/projects/1d21bc5503ba476caaff6898434c5c52/users/f95021839c22479abd5995687aa7992c/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 448 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:18.101456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532365-69fb-4df6-bd9f-69e4eb09cf43 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.102321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532365-69fb-4df6-bd9f-69e4eb09cf43 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.103851 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06532365-69fb-4df6-bd9f-69e4eb09cf43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.168657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-06532365-69fb-4df6-bd9f-69e4eb09cf43 None None] MFA Rules not processed for user `6cdd0a7f3f12414ea85ebffc9d531773`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:18.171830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-06532365-69fb-4df6-bd9f-69e4eb09cf43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:18.325130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 878/1742] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:18.333954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:18.334673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.335022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.335303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.342313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.342654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.342944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.346223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:18.374007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:18.444730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] MFA Rules not processed for user `f95021839c22479abd5995687aa7992c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:18.448696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:18.512766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 865/1743] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:17 2022] POST /identity/v3/projects => generated 405 bytes in 641 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:18.517513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:18.530300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:18.607179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e67265-fcff-48ca-8890-ef86b3901763 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:18.613800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:18.614500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.614784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.615060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.615765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:18.618564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:18.706822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 866/1744] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:18 2022] POST /identity/v3/users => generated 309 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:18.708857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:18.734655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6a62eb-df9d-4a83-9458-c837653299b6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:18.744170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:18.745195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.745489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.745765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.746768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:18.765733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:18.781096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 879/1745] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:18 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 455 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:18.795618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 867/1746] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:18 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:18.800160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:18.803091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:18.805384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:18.815904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-024544ed-ca22-4ab2-9d9c-6819607450e8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:18.825390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:18.826508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.826800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.827256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] PATH_INFO: `/v3/projects/c77168e7b9df4bc9b6dfe7d4421b2414/users/1594d5054b9247218f03ab5ee5eb5cf0/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:18.850626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77168e7b9df4bc9b6dfe7d4421b2414, user_id=1594d5054b9247218f03ab5ee5eb5cf0, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:18.862112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:18.972437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f954c9f9-1f1f-42ad-a7a0-f7d0a2054820 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:18.983834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:18.984926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:18.986471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:18.986891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.003393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:19.022201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 880/1747] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:18 2022] PUT /identity/v3/projects/c77168e7b9df4bc9b6dfe7d4421b2414/users/1594d5054b9247218f03ab5ee5eb5cf0/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:19.023396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:19.218124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:19.224060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:19.246647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 868/1748] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:18 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 445 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:19.259072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4231532c-124b-45a5-b29e-6a25e5321909 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:19.274812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:19.275525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.275812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.276119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.277559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:19.279222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:19.301066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:19.311262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 881/1749] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:19 2022] GET /identity/v3/roles => generated 1420 bytes in 288 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:19.315603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:19.317204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2c89d7-a70c-48b8-8316-d14ea7943bf0 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:19.332663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:19.332907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41452aeb-6522-4541-b8d6-c1ccef873b57 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:19.333684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.334146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.334444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] PATH_INFO: `/v3/users/19ebdf8423054a94b71d12990fe56445` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.335707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=19ebdf8423054a94b71d12990fe56445)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:19.338235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:19.339105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0df6da-df75-4a1b-8671-4bf821ba5c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:19.340235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df6da-df75-4a1b-8671-4bf821ba5c51 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.340692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df6da-df75-4a1b-8671-4bf821ba5c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.341112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0df6da-df75-4a1b-8671-4bf821ba5c51 admin admin] PATH_INFO: `/v3/projects/c77168e7b9df4bc9b6dfe7d4421b2414/users/1594d5054b9247218f03ab5ee5eb5cf0/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.344138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df6da-df75-4a1b-8671-4bf821ba5c51 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c77168e7b9df4bc9b6dfe7d4421b2414, user_id=1594d5054b9247218f03ab5ee5eb5cf0, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:19.350170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0df6da-df75-4a1b-8671-4bf821ba5c51 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:19.369599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 882/1750] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:19 2022] PUT /identity/v3/projects/c77168e7b9df4bc9b6dfe7d4421b2414/users/1594d5054b9247218f03ab5ee5eb5cf0/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:19.392603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.393161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.393617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.441912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] MFA Rules not processed for user `1594d5054b9247218f03ab5ee5eb5cf0`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:19.444346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:19.564881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 883/1751] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:19.600128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:19.606038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:19.617987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '19ebdf8423054a94b71d12990fe56445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:19.618960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '19ebdf8423054a94b71d12990fe56445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:19.724428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a1e878-88f9-446b-bee2-6d4442e028d4 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:19.730149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3daa5185-8f76-49c4-9788-6e3b6663887a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:19.731005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5185-8f76-49c4-9788-6e3b6663887a service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.736353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '19ebdf8423054a94b71d12990fe56445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:19.746452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '19ebdf8423054a94b71d12990fe56445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:19.748177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5185-8f76-49c4-9788-6e3b6663887a service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.750798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3daa5185-8f76-49c4-9788-6e3b6663887a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.779085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa5185-8f76-49c4-9788-6e3b6663887a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:19.781527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2abc6d92-1923-443e-817e-4281f63184b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '19ebdf8423054a94b71d12990fe56445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:19.786569 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3daa5185-8f76-49c4-9788-6e3b6663887a service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:19.816156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 869/1752] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:19 2022] DELETE /identity/v3/users/19ebdf8423054a94b71d12990fe56445 => generated 0 bytes in 515 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:19.817501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e2a5d-b3a6-481e-812c-082c29c665a5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.817859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e2a5d-b3a6-481e-812c-082c29c665a5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.818206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-290e2a5d-b3a6-481e-812c-082c29c665a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.864864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 884/1753] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:19 2022] GET /identity/v3/auth/tokens => generated 3339 bytes in 264 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:19.868995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:19.873932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:19.875891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:19.922125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-290e2a5d-b3a6-481e-812c-082c29c665a5 None None] MFA Rules not processed for user `fe46d7bd88c449fda863c0a82a2ed8cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:19.923800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-290e2a5d-b3a6-481e-812c-082c29c665a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:19.965874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:19.968532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:20.018805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:20.108270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 870/1754] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3339 bytes in 292 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:20.110017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90360e06-5668-4712-bf87-38228699a619 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:20.110504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90360e06-5668-4712-bf87-38228699a619 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:20.112367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90360e06-5668-4712-bf87-38228699a619 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:20.128473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 885/1755] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:20.142975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:20.150615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:20.173289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-90360e06-5668-4712-bf87-38228699a619 None None] MFA Rules not processed for user `13d8c04611e24dffad16f8bb97e111f2`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:20.188753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-90360e06-5668-4712-bf87-38228699a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:20.349506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd96c55d-df5f-46bd-910a-d93f59bd4015 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:20.358119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 871/1756] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:20 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:20.370105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90360e06-5668-4712-bf87-38228699a619 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:20.370687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe017ea-e676-4e2b-8430-e1342823397d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:20.370687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe017ea-e676-4e2b-8430-e1342823397d service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:20.370687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe017ea-e676-4e2b-8430-e1342823397d service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:20.370687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe017ea-e676-4e2b-8430-e1342823397d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:20.371023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-90360e06-5668-4712-bf87-38228699a619 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:20.371103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:20.391645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe017ea-e676-4e2b-8430-e1342823397d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:20.395169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe017ea-e676-4e2b-8430-e1342823397d service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:20.411524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90360e06-5668-4712-bf87-38228699a619 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:20.425958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:20.427147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:20.428134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:20.428806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] PATH_INFO: `/v3/users/9ad22a72fbb345309f5f276babd350d3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:20.431233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 886/1757] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:20.441161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] RBAC: Authorizing `identity:delete_user(user_id=9ad22a72fbb345309f5f276babd350d3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:20.444276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:20.797703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9ad22a72fbb345309f5f276babd350d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:20.805035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9ad22a72fbb345309f5f276babd350d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:20.919292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9ad22a72fbb345309f5f276babd350d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:20.927523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9ad22a72fbb345309f5f276babd350d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:20.961822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9ad22a72fbb345309f5f276babd350d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:20.974308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 872/1758] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:20 2022] DELETE /identity/v3/users/9ad22a72fbb345309f5f276babd350d3 => generated 0 bytes in 615 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:20.978278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635c1d-7eeb-43a7-a61e-3c846257edb7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:20.978788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635c1d-7eeb-43a7-a61e-3c846257edb7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:20.979096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6635c1d-7eeb-43a7-a61e-3c846257edb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:21.002460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b6635c1d-7eeb-43a7-a61e-3c846257edb7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:21.003936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b6635c1d-7eeb-43a7-a61e-3c846257edb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:21.159853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 887/1759] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:21.187243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:21.189256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:21.238462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:21.238760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:21.239042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:21.280643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] MFA Rules not processed for user `97ce6d944eb74b9794d6d0d5f18d4bad`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:21.281880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:21.396978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1df5111c-6afa-4754-9303-1436a0e26bec admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:21.421040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-541ea579-4b3a-498b-93da-ad313e5d47ed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:21.421713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ea579-4b3a-498b-93da-ad313e5d47ed service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:21.425793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ea579-4b3a-498b-93da-ad313e5d47ed service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:21.426084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-541ea579-4b3a-498b-93da-ad313e5d47ed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:21.458332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:21.461236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ea579-4b3a-498b-93da-ad313e5d47ed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:21.462793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-541ea579-4b3a-498b-93da-ad313e5d47ed service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:21.486890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 888/1760] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 250 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:21.496568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 873/1761] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:21 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:21.913588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:21.915430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:21.939638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f2aca2-f228-4709-870d-459a04e2c3ad None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:21.972682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:21.973650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:21.974003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:21.974289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] PATH_INFO: `/v3/projects/54db129703ce4fafb33196fda2ff3866` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:21.980921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] RBAC: Authorizing `identity:delete_project(project_id=54db129703ce4fafb33196fda2ff3866)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:21.983404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:22.013051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:22.214637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8a42-defa-48c9-b47e-2cdbd702b55f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:22.215001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8a42-defa-48c9-b47e-2cdbd702b55f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:22.215340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eecc8a42-defa-48c9-b47e-2cdbd702b55f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:22.245671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '54db129703ce4fafb33196fda2ff3866', 'request_id': 'req-907f3705-1a55-4c27-9285-fe235ea0e656'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:22.255714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '54db129703ce4fafb33196fda2ff3866', 'request_id': 'req-907f3705-1a55-4c27-9285-fe235ea0e656'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:22.259362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-eecc8a42-defa-48c9-b47e-2cdbd702b55f None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:22.262390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-eecc8a42-defa-48c9-b47e-2cdbd702b55f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:22.280798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '54db129703ce4fafb33196fda2ff3866', 'request_id': 'req-907f3705-1a55-4c27-9285-fe235ea0e656'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:22.361512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:22.396770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] The token cache is being invalidate because project 54db129703ce4fafb33196fda2ff3866 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:22.397282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:22.398600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:22.401291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 889/1762] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:21 2022] DELETE /identity/v3/projects/54db129703ce4fafb33196fda2ff3866 => generated 0 bytes in 488 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:22.406167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:22.408039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:22.425192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 874/1763] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:56:22 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:22.489341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-907f3705-1a55-4c27-9285-fe235ea0e656 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:22.504207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:22.505554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:22.506176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:22.506614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] PATH_INFO: `/v3/users/f95021839c22479abd5995687aa7992c` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:22.508241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f95021839c22479abd5995687aa7992c)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:22.515524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:22.625431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9484ae-9359-448d-83a6-d9b89aa1a8f6 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:22.626146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9484ae-9359-448d-83a6-d9b89aa1a8f6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:22.626453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9484ae-9359-448d-83a6-d9b89aa1a8f6 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:22.627915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 875/1764] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:56:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:22.647661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:22.648418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:22.649517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:22.713356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:22.724035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:22.914909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f95021839c22479abd5995687aa7992c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:22.921641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f95021839c22479abd5995687aa7992c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:23.028653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 876/1765] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:56:22 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 382 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:23.029823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:23.033807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:23.061401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f95021839c22479abd5995687aa7992c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:23.078660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f95021839c22479abd5995687aa7992c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:23.108986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f95021839c22479abd5995687aa7992c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:23.179935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 890/1766] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:22 2022] DELETE /identity/v3/users/f95021839c22479abd5995687aa7992c => generated 0 bytes in 771 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:23.184292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:23.189136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:23.305219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8599ab-5fd9-4be8-b842-bd947c156694 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:23.473726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be144c7f-4aee-4039-bc03-4fffc2ae38d8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:23.507841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:23.519164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:23.521148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:23.523745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:23.524138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:23.532122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:23.586430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:23.593602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:23.593980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:23.595251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:23.600222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:23.602693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:23.612362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:23.629385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 891/1767] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:23 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 449 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:23.755552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:23.757367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:23.769016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 877/1768] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:56:23 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 740 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:56:24.064606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:24.974982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:24.977639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:25.125429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3314f0c3-e4f8-48e0-9e19-9a4e0bf14d7d service placement] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:25.125429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:25.125429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:25.125429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:25.126566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] PATH_INFO: `/v3/projects/ed2f14ef34524242a67069f619fad5c2` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:25.126566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed2f14ef34524242a67069f619fad5c2)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:25.126566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:25.396531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed2f14ef34524242a67069f619fad5c2', 'request_id': 'req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:25.444241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:25.470382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed2f14ef34524242a67069f619fad5c2', 'request_id': 'req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:25.486700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed2f14ef34524242a67069f619fad5c2', 'request_id': 'req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:25.576940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] The token cache is being invalidate because project ed2f14ef34524242a67069f619fad5c2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:25.578226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:25.580125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:25.582273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 892/1769] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:24 2022] DELETE /identity/v3/projects/ed2f14ef34524242a67069f619fad5c2 => generated 0 bytes in 609 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:25.586897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:25.609574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-291b3d89-17fa-4a14-8458-f927beacaafa service swift] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:25.784487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:25.785672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:25.785984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:25.786496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] PATH_INFO: `/v3/users/1594d5054b9247218f03ab5ee5eb5cf0` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:25.789884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] RBAC: Authorizing `identity:delete_user(user_id=1594d5054b9247218f03ab5ee5eb5cf0)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:25.793677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:25.937219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:25.938714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:26.003104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d97f86-2cda-4eff-bcb0-c92d436cc43d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:26.025902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1594d5054b9247218f03ab5ee5eb5cf0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.026917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1594d5054b9247218f03ab5ee5eb5cf0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.036618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:26.037990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:26.038430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:26.038869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] PATH_INFO: `/v3/users/72b04b1ecc214237900350ae7867daab` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:26.040689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] RBAC: Authorizing `identity:delete_user(user_id=72b04b1ecc214237900350ae7867daab)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:26.057590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:26.141054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1594d5054b9247218f03ab5ee5eb5cf0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.150541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1594d5054b9247218f03ab5ee5eb5cf0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.182010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9d859190-e72f-4482-8de6-495ac076cf5a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1594d5054b9247218f03ab5ee5eb5cf0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.199477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 878/1770] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:25 2022] DELETE /identity/v3/users/1594d5054b9247218f03ab5ee5eb5cf0 => generated 0 bytes in 614 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:26.274285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:26.429697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:26.429992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:26.432023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:26.448637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '72b04b1ecc214237900350ae7867daab'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.451969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '72b04b1ecc214237900350ae7867daab'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.483033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] MFA Rules not processed for user `516b221cb1794430befadcde5f9fd77b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:26.494191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:26.563598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '72b04b1ecc214237900350ae7867daab'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.572713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '72b04b1ecc214237900350ae7867daab'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.590164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '72b04b1ecc214237900350ae7867daab'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:26.598196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 893/1771] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:25 2022] DELETE /identity/v3/users/72b04b1ecc214237900350ae7867daab => generated 0 bytes in 662 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:26.599671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:26.602636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:26.688771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 879/1772] 10.222.0.13 () {58 vars in 1015 bytes} [Thu Jun 16 11:56:26 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 260 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:26.822656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b5782db-48ed-4ff6-9ca5-3109e4a05037 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:26.830776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:26.831591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:26.833341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:26.833664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] PATH_INFO: `/v3/projects/1d21bc5503ba476caaff6898434c5c52` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:26.835314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d21bc5503ba476caaff6898434c5c52)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:26.837436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:26.989731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d21bc5503ba476caaff6898434c5c52', 'request_id': 'req-95f1eb47-95a1-4488-a2fb-b772f809dd85'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.002206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d21bc5503ba476caaff6898434c5c52', 'request_id': 'req-95f1eb47-95a1-4488-a2fb-b772f809dd85'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.010303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d21bc5503ba476caaff6898434c5c52', 'request_id': 'req-95f1eb47-95a1-4488-a2fb-b772f809dd85'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.068360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:27.071164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:27.088498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e301b246-3753-4d24-8998-eb42f0a9cc42 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:27.127338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] The token cache is being invalidate because project 1d21bc5503ba476caaff6898434c5c52 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:27.131257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.131257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.131257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 894/1773] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:26 2022] DELETE /identity/v3/projects/1d21bc5503ba476caaff6898434c5c52 => generated 0 bytes in 532 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:27.135623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:27.137370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:27.295289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f1eb47-95a1-4488-a2fb-b772f809dd85 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:27.307441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:27.309094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:27.309543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:27.310129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:27.314479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:27.329134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:27.331257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac60614e-6c51-4601-9055-07d2845edfae service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:27.333044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:27.333761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:27.339852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:27.339852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:27.354020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:27.355507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:27.365937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 895/1774] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 234 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:27.371698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:27.372031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:27.373728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:27.391771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 880/1775] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 322 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:27.434373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:27.435916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:27.444018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:27.453240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:27.455946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:27.603496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 896/1776] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 238 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:27.605775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:27.607617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:27.623682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c83bb1d-28f8-43f8-b6f0-d909cf55db28 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:27.629615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:27.636609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:27.636609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:27.636609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] PATH_INFO: `/v3/projects/c77168e7b9df4bc9b6dfe7d4421b2414` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:27.636609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c77168e7b9df4bc9b6dfe7d4421b2414)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:27.636609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:27.710144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcc1ec2-2b8d-471b-a1e6-2288ee57dcbe service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:27.739118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:27.741052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:27.741629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:27.742130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] PATH_INFO: `/v3/projects/3d978da939764925bca041e365a1cf5f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:27.744009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d978da939764925bca041e365a1cf5f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:27.746547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:27.893267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c77168e7b9df4bc9b6dfe7d4421b2414', 'request_id': 'req-1cd9b28d-8110-47bc-bd26-487ce93d4042'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.911543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c77168e7b9df4bc9b6dfe7d4421b2414', 'request_id': 'req-1cd9b28d-8110-47bc-bd26-487ce93d4042'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.930069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c77168e7b9df4bc9b6dfe7d4421b2414', 'request_id': 'req-1cd9b28d-8110-47bc-bd26-487ce93d4042'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.961124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d978da939764925bca041e365a1cf5f', 'request_id': 'req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:27.995721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d978da939764925bca041e365a1cf5f', 'request_id': 'req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.053615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] The token cache is being invalidate because project c77168e7b9df4bc9b6dfe7d4421b2414 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:28.053928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.054737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1cd9b28d-8110-47bc-bd26-487ce93d4042 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.061519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 897/1777] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:27 2022] DELETE /identity/v3/projects/c77168e7b9df4bc9b6dfe7d4421b2414 => generated 0 bytes in 453 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:28.064150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:28.064606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:28.075033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:28.080973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:28.122273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:28.123582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:28.166293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d978da939764925bca041e365a1cf5f', 'request_id': 'req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.198230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] The token cache is being invalidate because project 3d978da939764925bca041e365a1cf5f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:28.198869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.200583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1219ca8a-374c-4d90-8892-ccdeb82dfbfa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.206032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 881/1778] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:27 2022] DELETE /identity/v3/projects/3d978da939764925bca041e365a1cf5f => generated 0 bytes in 754 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:28.207646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:28.213551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:28.213551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:28.275569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:28.282391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:28.305411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 898/1779] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:28.306903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:28.310186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:28.313301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 882/1780] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:28.314385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:28.352610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3540641b-ff72-4126-9509-9b2f11daca61 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:28.352822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5675ffd3-a87e-4d7f-a372-89e43669109c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:28.372994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:28.374502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:28.376071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:28.380106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:28.380225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:28.380357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:28.380471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:28.380575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] PATH_INFO: `/v3/users/fe46d7bd88c449fda863c0a82a2ed8cf` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:28.380657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:28.380657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:28.380876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fe46d7bd88c449fda863c0a82a2ed8cf)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:28.382933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:28.387486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18758105-3bd6-44b1-bb31-37d000291fdc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:28.423780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 883/1781] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:28.426891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:28.428615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:28.430509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:28.517020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:28.518398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:28.553765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 884/1782] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:28.555060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:28.577404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ecc905f-0ec8-428c-823a-af303a5e2122 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:28.583187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:28.584634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:28.586344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:28.586701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:28.588357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:28.595133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:28.698855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fe46d7bd88c449fda863c0a82a2ed8cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.699593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fe46d7bd88c449fda863c0a82a2ed8cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.748972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 885/1783] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:28 2022] POST /identity/v3/projects => generated 385 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:28.751460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:28.794801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fe46d7bd88c449fda863c0a82a2ed8cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.805624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fe46d7bd88c449fda863c0a82a2ed8cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.828356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fe46d7bd88c449fda863c0a82a2ed8cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:28.839582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 899/1784] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:28 2022] DELETE /identity/v3/users/fe46d7bd88c449fda863c0a82a2ed8cf => generated 0 bytes in 531 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:28.839582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:28.846150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:28.952985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bc97d09-472b-4a39-a3f0-cbfd7afcc4dc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.024685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a609c1e9-4836-4c12-a974-bbcd4c703cf5 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.038372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.040984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.041542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.043497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.044441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.048815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.052068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:29.130933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.140238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.142505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.144111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.146803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:29.147249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.152888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.162795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 900/1785] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:28 2022] POST /identity/v3/users => generated 293 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:29.166499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:29.191988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 886/1786] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 442 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:29.199967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:29.204347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:29.205210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6986e-33f9-43ed-a9ec-f052edcb728b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.235776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.236892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.237351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.237761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.238628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.240423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.244863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c15a3c-2122-418c-9199-68ed5bfb1dde admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.252321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.253219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.253559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.253950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.256218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.259796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.269678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 901/1787] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:29.277235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:29.324362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97628e25-a3dd-43fa-bc69-4ebaeccf8bcb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.332835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.333800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.334233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.334685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.335861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.338252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.356793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 902/1788] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:29 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:29.358309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:29.360860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:29.381319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6717e05-b660-4b0f-bf43-45ab7d5f1ef2 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.400173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.401196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.401527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.401853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] PATH_INFO: `/v3/projects/e1bc6c309ff440dba31625b6083a6b60` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.405070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1bc6c309ff440dba31625b6083a6b60)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.407042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.438087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:29.447740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 887/1789] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:29 2022] POST /identity/v3/projects => generated 379 bytes in 252 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:29.449954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:29.638125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1bc6c309ff440dba31625b6083a6b60', 'request_id': 'req-a164e88d-ffd7-4467-af53-8a8752d06ad9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:29.678045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1bc6c309ff440dba31625b6083a6b60', 'request_id': 'req-a164e88d-ffd7-4467-af53-8a8752d06ad9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:29.692672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1bc6c309ff440dba31625b6083a6b60', 'request_id': 'req-a164e88d-ffd7-4467-af53-8a8752d06ad9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:29.712225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83654b35-efc7-4303-91f0-d34edf3e1035 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.826633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] The token cache is being invalidate because project e1bc6c309ff440dba31625b6083a6b60 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:29.828579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:29.830281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:29.845169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 903/1790] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:29 2022] DELETE /identity/v3/projects/e1bc6c309ff440dba31625b6083a6b60 => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:29.845169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:29.851678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:29.862289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.873815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.873815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.873815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] PATH_INFO: `/v3/projects/aeae41f39a17486993101e097485aa0f/users/c3ea7153c0da47f6915e2c039ef72a12/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.874164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a164e88d-ffd7-4467-af53-8a8752d06ad9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:29.888698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a023ad22-fc88-4f24-aa16-4d9d30610a66 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:29.889746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023ad22-fc88-4f24-aa16-4d9d30610a66 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:29.890152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023ad22-fc88-4f24-aa16-4d9d30610a66 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:29.890586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a023ad22-fc88-4f24-aa16-4d9d30610a66 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:29.891416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023ad22-fc88-4f24-aa16-4d9d30610a66 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.893678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a023ad22-fc88-4f24-aa16-4d9d30610a66 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:29.928350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] RBAC: Authorizing `identity:create_grant(project_id=aeae41f39a17486993101e097485aa0f, user_id=c3ea7153c0da47f6915e2c039ef72a12, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:29.934657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:30.024915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 904/1791] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:29 2022] POST /identity/v3/users => generated 290 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:30.032117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048d619-64ca-4b6e-baf3-9d57167c81d8 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.032617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048d619-64ca-4b6e-baf3-9d57167c81d8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.033570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7048d619-64ca-4b6e-baf3-9d57167c81d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.048949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 888/1792] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:29 2022] PUT /identity/v3/projects/aeae41f39a17486993101e097485aa0f/users/c3ea7153c0da47f6915e2c039ef72a12/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 601 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:30.058671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:30.101511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7048d619-64ca-4b6e-baf3-9d57167c81d8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:30.104254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7048d619-64ca-4b6e-baf3-9d57167c81d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:30.256441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6bdeae1-7fed-4e3e-aedd-ae2164327dfd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:30.266351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc410fd-7c7e-4edb-a07f-da874849be63 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:30.267246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc410fd-7c7e-4edb-a07f-da874849be63 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.267365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc410fd-7c7e-4edb-a07f-da874849be63 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.267631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc410fd-7c7e-4edb-a07f-da874849be63 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.268410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc410fd-7c7e-4edb-a07f-da874849be63 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:30.279891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc410fd-7c7e-4edb-a07f-da874849be63 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:30.280375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 905/1793] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:30.282244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.282543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.282841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.296561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 889/1794] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:30.298183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.298651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.299305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.329509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:30.331799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:30.335932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] MFA Rules not processed for user `c3ea7153c0da47f6915e2c039ef72a12`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:30.338723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:30.347350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 890/1795] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:30.350315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:30.387097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6614d167-8c62-4326-acfb-315b6c84166e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:30.399846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:30.402770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.403848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.405436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.406423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:30.408971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:30.435931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 891/1796] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:30 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:30.441560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:30.459048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aca3394a-d077-48b8-ab5a-5141049d4224 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:30.468748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:30.469782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.470254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.470835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.472488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:30.473390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:30.475860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:30.533595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 892/1797] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:30.534823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:30.537630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:30.551445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 906/1798] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 270 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:30.555999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:30.575010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43f7700c-8810-4e48-b81b-8cf423bc06bc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:30.581149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:30.582320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.582713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.583095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] PATH_INFO: `/v3/projects/fdb6d472385f4d198dc532b006fd711b/users/6b7b25f9563745cd832aa68d253ad80d/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.601556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60040de-7661-4cda-96c2-ba6a08497cec None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:30.615198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdb6d472385f4d198dc532b006fd711b, user_id=6b7b25f9563745cd832aa68d253ad80d, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:30.617579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a89f752f-674f-4484-9bde-ec4626b17fd1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:30.619493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:30.621125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f752f-674f-4484-9bde-ec4626b17fd1 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.621481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f752f-674f-4484-9bde-ec4626b17fd1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.621752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a89f752f-674f-4484-9bde-ec4626b17fd1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.625846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f752f-674f-4484-9bde-ec4626b17fd1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:30.630904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a89f752f-674f-4484-9bde-ec4626b17fd1 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:30.673653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:30.717030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 893/1799] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:30 2022] PUT /identity/v3/projects/fdb6d472385f4d198dc532b006fd711b/users/6b7b25f9563745cd832aa68d253ad80d/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:30.718004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:30.730682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:30.780158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 907/1800] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/projects => generated 365 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:30.781363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:30.781659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:30.781949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:30.852957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] MFA Rules not processed for user `6b7b25f9563745cd832aa68d253ad80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:30.854197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:30.929416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d67f5a15-6c6a-4ca0-86ac-899b03d6d226 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:30.998083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 908/1801] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 218 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:31.002579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.083042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.085907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.086196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.087588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.091406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.094753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ade48e52-69cc-4a13-b539-2bf0472d1450 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.098560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.102632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.103518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.103911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.104260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.105405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.107446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.109234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:31.182815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 909/1802] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/users => generated 283 bytes in 178 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:31.182815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.210374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-946f28b6-c2c6-47d1-b50d-10b61792cb94 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.222812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.222812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.222812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.222812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.223664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.224817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.299154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 894/1803] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:30 2022] POST /identity/v3/projects => generated 387 bytes in 582 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:31.300323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.389363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 910/1804] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:31 2022] POST /identity/v3/projects => generated 381 bytes in 212 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:31.390344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.455899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af087637-a985-4c98-a6fe-6e41ab157620 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.594306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.595096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.595550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.595835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.596663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.599267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.606244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a77a7ab8-e853-4111-a4f1-fcad9bd7162c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.611416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.617045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.617380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.617670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.618460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.620691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.624055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 895/1805] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 324 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:31.625233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.637268 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d3b51a8-c70b-42d0-a298-8d191807ba64 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.644277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.645175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.646005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.646134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.646986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.649261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.745822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 911/1806] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:31 2022] POST /identity/v3/users => generated 300 bytes in 355 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:31.754790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.771153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 896/1807] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:31 2022] POST /identity/v3/users => generated 297 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:31.774040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.809314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53ed2343-7aa9-4142-8a04-ef9124589a51 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.811776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.812791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.813161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a0a1289-9ce7-42c3-b682-bda6f606fb53 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.813582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.814059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.815165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.817973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.823852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.824732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.825086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.826445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.833811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 912/1808] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:31.834874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.839939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.844516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.860496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 897/1809] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 87 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:31.865718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9cb05b7-8729-4454-b616-63453ad8ee1e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.865883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.880603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.881481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.881783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.882068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.882229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b1e1002-3726-46ee-aa47-fa6cca6380c1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.883130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.884978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.890095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.891485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.892151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.892525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] PATH_INFO: `/v3/projects/c637612376914a3ba9c0554e04072232/users/cf00e855ea244a93ad658a5dc3c865a1/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.923687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 913/1810] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:31 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:31.925453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:31.950149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c637612376914a3ba9c0554e04072232, user_id=cf00e855ea244a93ad658a5dc3c865a1, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.953506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:31.956530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72620982-e8b8-4cbc-91fc-37fd1dd861cf admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:31.965694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d0d7729-a3f6-44b8-92c1-31ecba9e5574 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:31.967357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7729-a3f6-44b8-92c1-31ecba9e5574 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:31.967645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7729-a3f6-44b8-92c1-31ecba9e5574 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:31.968066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d0d7729-a3f6-44b8-92c1-31ecba9e5574 admin admin] PATH_INFO: `/v3/projects/64a5357f70f04cbe85362abfc5613233/users/31fd2029a35448148612acce1bd6b083/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:31.986092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d7729-a3f6-44b8-92c1-31ecba9e5574 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a5357f70f04cbe85362abfc5613233, user_id=31fd2029a35448148612acce1bd6b083, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:31.988828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d0d7729-a3f6-44b8-92c1-31ecba9e5574 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:32.037576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 898/1811] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:31 2022] PUT /identity/v3/projects/c637612376914a3ba9c0554e04072232/users/cf00e855ea244a93ad658a5dc3c865a1/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:32.042717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:32.042826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=9,keystone:SELECT=90,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:32.055184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 914/1812] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:31 2022] PUT /identity/v3/projects/64a5357f70f04cbe85362abfc5613233/users/31fd2029a35448148612acce1bd6b083/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:32.063527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:32.064585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:32.065326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:32.136283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] MFA Rules not processed for user `cf00e855ea244a93ad658a5dc3c865a1`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:32.137935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:32.214794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8b65171-5c76-4e0c-a7c8-8c297934def7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:32.347825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 915/1813] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:32 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:32.349535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:32.392533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:32.392533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:32.392533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:32.392533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] PATH_INFO: `/v3/projects/217b375272c44813ace2925ad44e0e5e/users/069b71ff50c8465e879e2616ae382b8b/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:32.452052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] RBAC: Authorizing `identity:create_grant(project_id=217b375272c44813ace2925ad44e0e5e, user_id=069b71ff50c8465e879e2616ae382b8b, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:32.458333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:32.459906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:32.544160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 899/1814] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:32 2022] PUT /identity/v3/projects/217b375272c44813ace2925ad44e0e5e/users/069b71ff50c8465e879e2616ae382b8b/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:32.556892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:32.558040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9495db5-e1fc-4739-9866-6997a5b81379 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:32.738600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34fc6af9-22ec-410a-8370-95ed8fff4e7e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:32.748187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7bc54-b282-49a1-8040-6c094dcdcf67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:32.748315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:32.748643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:32.748932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:32.749282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:32.750113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:32.752380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:32.753077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7bc54-b282-49a1-8040-6c094dcdcf67 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:32.753392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7bc54-b282-49a1-8040-6c094dcdcf67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:32.756523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7bc54-b282-49a1-8040-6c094dcdcf67 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:32.758294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7bc54-b282-49a1-8040-6c094dcdcf67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:32.762971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7bc54-b282-49a1-8040-6c094dcdcf67 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:32.804000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 916/1815] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 455 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:32.805398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ef4a4-b639-49e4-bd57-15e3f7810080 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:32.805693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ef4a4-b639-49e4-bd57-15e3f7810080 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:32.805973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-593ef4a4-b639-49e4-bd57-15e3f7810080 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:32.812776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 917/1816] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:56:32 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:32.813837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ef4a4-b639-49e4-bd57-15e3f7810080 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:32.860386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-593ef4a4-b639-49e4-bd57-15e3f7810080 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:32.876358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:32.877412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:32.877795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:32.878125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:32.883799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:32.885925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:32.889248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 900/1817] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:32 2022] POST /identity/v3/projects => generated 365 bytes in 344 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:32.890367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:32.916746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 918/1818] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:32 2022] GET /identity/v3/roles => generated 1420 bytes in 100 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:32.916746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:32.934774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e3b9352-4c16-400d-b448-d878dd68fec5 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:32.957044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:33.068774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e99b6a36-7c3f-4ff6-ad7a-4e4696e96a7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:33.071980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b6a36-7c3f-4ff6-ad7a-4e4696e96a7b admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.072451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b6a36-7c3f-4ff6-ad7a-4e4696e96a7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.072847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e99b6a36-7c3f-4ff6-ad7a-4e4696e96a7b admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.073849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b6a36-7c3f-4ff6-ad7a-4e4696e96a7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:33.077990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e99b6a36-7c3f-4ff6-ad7a-4e4696e96a7b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:33.098177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9dc2ab8-702b-4111-b5e0-fdd0cdd41af3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:33.118615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:33.119629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.119941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.120231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] PATH_INFO: `/v3/projects/64a5357f70f04cbe85362abfc5613233/users/31fd2029a35448148612acce1bd6b083/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.143802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64a5357f70f04cbe85362abfc5613233, user_id=31fd2029a35448148612acce1bd6b083, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:33.147177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:33.201870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 901/1819] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:32 2022] PUT /identity/v3/projects/64a5357f70f04cbe85362abfc5613233/users/31fd2029a35448148612acce1bd6b083/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:33.205351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:33.245138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 919/1820] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:32 2022] POST /identity/v3/users => generated 289 bytes in 331 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:33.248570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.249112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.249661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.327409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:33.329546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:33.417936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:33.491227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eee164b7-939b-431b-89ec-991efbd9bd69 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:33.510052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e68d308d-6ffa-4aff-bc81-926288232d76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:33.511099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d308d-6ffa-4aff-bc81-926288232d76 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.516217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d308d-6ffa-4aff-bc81-926288232d76 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.516583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e68d308d-6ffa-4aff-bc81-926288232d76 admin admin] PATH_INFO: `/v3/projects/217b375272c44813ace2925ad44e0e5e/users/069b71ff50c8465e879e2616ae382b8b/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.522537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d308d-6ffa-4aff-bc81-926288232d76 admin admin] RBAC: Authorizing `identity:create_grant(project_id=217b375272c44813ace2925ad44e0e5e, user_id=069b71ff50c8465e879e2616ae382b8b, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:33.532089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e68d308d-6ffa-4aff-bc81-926288232d76 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:33.558381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 902/1821] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:33 2022] PUT /identity/v3/projects/217b375272c44813ace2925ad44e0e5e/users/069b71ff50c8465e879e2616ae382b8b/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 356 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:33.560072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103129f-ab85-4467-80a8-5ab98c6c122f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.560386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103129f-ab85-4467-80a8-5ab98c6c122f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.560720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5103129f-ab85-4467-80a8-5ab98c6c122f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.659336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5103129f-ab85-4467-80a8-5ab98c6c122f None None] MFA Rules not processed for user `31fd2029a35448148612acce1bd6b083`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:33.661403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5103129f-ab85-4467-80a8-5ab98c6c122f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:33.726028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 920/1822] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:56:33 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 480 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:33.731639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:33.773207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2edc61-b50e-4234-8f6c-2d01e1262cd9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:33.822408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 903/1823] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:33 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 263 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:33.825464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.826176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.826404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.879025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] MFA Rules not processed for user `069b71ff50c8465e879e2616ae382b8b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:33.880625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:33.889539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=83,keystone:DELETE=12,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:33.972371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:33.974452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:33.974452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:33.974452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:33.974786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:33.978211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.001598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 904/1824] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:33 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:34.002541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 921/1825] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:33 2022] GET /identity/v3/roles => generated 1420 bytes in 276 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:34.003116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.007042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:34.007444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.038812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93c50426-4f1c-4012-9e31-0a3e4f222554 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:34.059305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:34.061011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.061363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.061659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] PATH_INFO: `/v3/projects/c3ad7d438b2c45daa0d4ebef78702667/users/152bf6acced24405a93b40377e78f883/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.102434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dbd9afe-8655-4605-bbd8-a13527a7b16e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:34.105363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:34.106202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.106603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.107545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.111476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ad7d438b2c45daa0d4ebef78702667, user_id=152bf6acced24405a93b40377e78f883, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:34.115602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.118819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:34.120751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.128746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:34.134592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 905/1826] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:34 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:34.137332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.139351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:34.166657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138064de-62d4-44d5-977b-8b89e5e7a8d2 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:34.180727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:34.181926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.182277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.182562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.204902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 922/1827] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:34 2022] PUT /identity/v3/projects/c3ad7d438b2c45daa0d4ebef78702667/users/152bf6acced24405a93b40377e78f883/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:34.209982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.210215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:34.211881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.231295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 906/1828] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:34 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:34.266169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.305252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7c521c-1a10-42ee-97bf-434e690d70e3 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:34.362122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6067fc-858f-4364-93ae-d3d6b281d0d5 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:34.367574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:34.370781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.370781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.370781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.370781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:34.373958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.390312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 923/1829] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:34 2022] GET /identity/v3/roles => generated 1420 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:34.391606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.405054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be424ec-9171-4ef0-85f3-d3c418e35569 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:34.414993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d14e0cb-8c27-41de-ab0f-e08003b9c2e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:34.414993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e0cb-8c27-41de-ab0f-e08003b9c2e8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.414993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e0cb-8c27-41de-ab0f-e08003b9c2e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.414993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d14e0cb-8c27-41de-ab0f-e08003b9c2e8 admin admin] PATH_INFO: `/v3/projects/c3ad7d438b2c45daa0d4ebef78702667/users/152bf6acced24405a93b40377e78f883/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.417898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14e0cb-8c27-41de-ab0f-e08003b9c2e8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c3ad7d438b2c45daa0d4ebef78702667, user_id=152bf6acced24405a93b40377e78f883, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:34.421787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d14e0cb-8c27-41de-ab0f-e08003b9c2e8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.444426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:34.444426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.444426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.444426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] PATH_INFO: `/v3/users/c3ea7153c0da47f6915e2c039ef72a12` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.451576 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3ea7153c0da47f6915e2c039ef72a12)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:34.452282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 924/1830] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:34 2022] PUT /identity/v3/projects/c3ad7d438b2c45daa0d4ebef78702667/users/152bf6acced24405a93b40377e78f883/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 44 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:34.452282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:34.452538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:34.452713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:34.452713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:34.501506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] MFA Rules not processed for user `152bf6acced24405a93b40377e78f883`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:34.502981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:34.672882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 925/1831] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:34 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:34.674058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.743546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3ea7153c0da47f6915e2c039ef72a12'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:34.748789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3ea7153c0da47f6915e2c039ef72a12'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:34.844025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3ea7153c0da47f6915e2c039ef72a12'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:34.856225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3ea7153c0da47f6915e2c039ef72a12'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:34.883068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3ea7153c0da47f6915e2c039ef72a12'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:34.894499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 907/1832] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:34 2022] DELETE /identity/v3/users/c3ea7153c0da47f6915e2c039ef72a12 => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:34.895517 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:34.897107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:34.898738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915dd8cd-974b-449a-9802-407a9a45fdcb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:35.024021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:35.027787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.031190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.031454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.039063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.046349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ac0c68-b708-4c8c-b01e-e075fcbd83f8 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:35.052939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:35.054402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.054614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.055145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.058297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.087925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:35.090367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:35.110672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 908/1833] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:34 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 215 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:35.112439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:35.114212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.137301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc161b81-de09-4ff6-a8a0-a9e361e448b6 service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:35.142612 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:35.143588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.144120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.144736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.148677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.336359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:35.349528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35e50118-f09b-48b8-a84f-c847fa8ba4e9 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:35.361367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:35.363688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:35.370165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb13a99-7439-4cc2-837d-f6910f82387b service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:35.392794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 926/1834] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:34 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 719 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:35.395463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.395765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.399405 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 909/1835] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:35 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 287 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:35.404539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.404757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e8177-9db5-4ff9-8745-89c2357b9211 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.405224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e8177-9db5-4ff9-8745-89c2357b9211 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.408082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062e8177-9db5-4ff9-8745-89c2357b9211 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.411320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 910/1836] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:56:35 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:35.454852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:35.457123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.483255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.483832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.484366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.543461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:35.547846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.597741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 911/1837] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:56:35 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:35.619513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:35.677816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 927/1838] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:35 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 284 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:35.693702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:35.697275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.817176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c8c971a-0f78-4c23-8a28-ea8bd8d6af26 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:35.821219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:35.821890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:35.822179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:35.822465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:35.844582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:35.854264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:35.885814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 912/1839] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:35 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:35.970758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:35.972468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:35.990318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:35.995502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfaa2949-0969-447b-ac06-16024f38ca21 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:36.006354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:36.007038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:36.009720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:36.010073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:36.010769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efbbd5c7-ad1e-4112-a2ee-d4a57290a069 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:36.014752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:36.030340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:36.031129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:36.031420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:36.031700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:36.085568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:36.088282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:36.122856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 913/1840] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:35 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 132 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:36.244200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:36.246079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:36.306452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 928/1841] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:35 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:37.115685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:37.432088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:37.613337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:37.615604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:37.664318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91f3d250-57e3-41f3-bcf3-80be21fff037 service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:37.682846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:37.684503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:37.684967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:37.691924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] PATH_INFO: `/v3/projects/aeae41f39a17486993101e097485aa0f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:37.691924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] RBAC: Authorizing `identity:delete_project(project_id=aeae41f39a17486993101e097485aa0f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:37.696786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:37.947190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'aeae41f39a17486993101e097485aa0f', 'request_id': 'req-ba71f2f5-268a-497b-8765-572724d6512d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:37.966903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'aeae41f39a17486993101e097485aa0f', 'request_id': 'req-ba71f2f5-268a-497b-8765-572724d6512d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.234764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'aeae41f39a17486993101e097485aa0f', 'request_id': 'req-ba71f2f5-268a-497b-8765-572724d6512d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.249348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] The token cache is being invalidate because project aeae41f39a17486993101e097485aa0f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:38.249709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.250435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.252366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 914/1842] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:37 2022] DELETE /identity/v3/projects/aeae41f39a17486993101e097485aa0f => generated 0 bytes in 639 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:38.256499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:38.268421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e0fcb76-7428-4e0c-b72e-82051e0af636 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:38.360394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:38.361502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:38.361950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:38.362394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] PATH_INFO: `/v3/users/31fd2029a35448148612acce1bd6b083` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:38.363915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] RBAC: Authorizing `identity:delete_user(user_id=31fd2029a35448148612acce1bd6b083)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:38.366611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:38.554743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '31fd2029a35448148612acce1bd6b083'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.555695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '31fd2029a35448148612acce1bd6b083'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.626273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '31fd2029a35448148612acce1bd6b083'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.634399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '31fd2029a35448148612acce1bd6b083'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.647421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '31fd2029a35448148612acce1bd6b083'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:38.660618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 929/1843] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:38 2022] DELETE /identity/v3/users/31fd2029a35448148612acce1bd6b083 => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:39.208710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:39.210423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:39.226958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba71f2f5-268a-497b-8765-572724d6512d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:39.309349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:39.310924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:39.311316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:39.311774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] PATH_INFO: `/v3/projects/64a5357f70f04cbe85362abfc5613233` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:39.313384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64a5357f70f04cbe85362abfc5613233)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:39.315712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:39.464020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64a5357f70f04cbe85362abfc5613233', 'request_id': 'req-59ca6e0b-2063-4173-8133-e19eba9fd1f0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.469113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:39.470691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:39.473157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64a5357f70f04cbe85362abfc5613233', 'request_id': 'req-59ca6e0b-2063-4173-8133-e19eba9fd1f0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.485891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d22fb833-d9b3-4cf0-9db6-9c82103866fa admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:39.541383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:39.542240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:39.542524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:39.542815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] PATH_INFO: `/v3/users/7274bea2ee0f4dd0a98580084c6cec2f` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:39.544612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7274bea2ee0f4dd0a98580084c6cec2f)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:39.546522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:39.586207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64a5357f70f04cbe85362abfc5613233', 'request_id': 'req-59ca6e0b-2063-4173-8133-e19eba9fd1f0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.599336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] The token cache is being invalidate because project 64a5357f70f04cbe85362abfc5613233 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:39.599629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.600299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59ca6e0b-2063-4173-8133-e19eba9fd1f0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.601858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 915/1844] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:39 2022] DELETE /identity/v3/projects/64a5357f70f04cbe85362abfc5613233 => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:39.607255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387def18-cf11-4de9-83c3-5131990402b6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:39.607548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387def18-cf11-4de9-83c3-5131990402b6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:39.607994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-387def18-cf11-4de9-83c3-5131990402b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:39.749038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:39.752891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-387def18-cf11-4de9-83c3-5131990402b6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:39.754481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-387def18-cf11-4de9-83c3-5131990402b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:39.762113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 916/1845] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 156 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:39.769526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:39.769802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:39.770078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:39.789066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:39.790264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:39.799057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 917/1846] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:39.803671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:39.813395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45a9285f-da85-4d7f-86c0-ef52bc7d3505 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:39.818034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:39.818748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:39.819201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:39.819601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:39.823768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:39.823768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:39.823768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:39.837966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 918/1847] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:39.841484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7274bea2ee0f4dd0a98580084c6cec2f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.842671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7274bea2ee0f4dd0a98580084c6cec2f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.843667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:39.867080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f74a133-3a42-4719-9c39-8ffb6d628e0d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:39.867080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:39.867080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:39.867080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:39.868193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:39.868193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:39.868193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:39.969210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7274bea2ee0f4dd0a98580084c6cec2f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.979882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7274bea2ee0f4dd0a98580084c6cec2f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:39.984282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 919/1848] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:39 2022] POST /identity/v3/projects => generated 387 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:39.984282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:39.995890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-5ddb7fa0-b569-4031-b18b-b8f3829c3535 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7274bea2ee0f4dd0a98580084c6cec2f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:40.002436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 930/1849] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:39 2022] DELETE /identity/v3/users/7274bea2ee0f4dd0a98580084c6cec2f => generated 0 bytes in 534 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:40.009835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.009835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.010143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.029612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:40.030953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:40.046611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e86dcb0-985a-47aa-9d26-d25ef461dc85 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.058999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:40.101662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 931/1850] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:40.106792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.107546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.107937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.108396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.109166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.111412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.112464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.113826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:40.171632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13348e82-7258-4c3e-9530-8df4fe6b592c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.175543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.176229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.176517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.176798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.185981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.187438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.196905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 932/1851] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:40.201006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 920/1852] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:39 2022] POST /identity/v3/users => generated 294 bytes in 218 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:40.204935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.206373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:40.220106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1943d076-11cb-4c62-9101-7c91c6d6d821 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.224977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.225645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.229748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.229748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.229748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.229748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.238895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 933/1853] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:40.244586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.254925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba54683a-2900-435b-9f15-f5d6ea6b1a59 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.265422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1485f25-1293-4ac2-bc28-8dc3ed38a690 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.266341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1485f25-1293-4ac2-bc28-8dc3ed38a690 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.266691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1485f25-1293-4ac2-bc28-8dc3ed38a690 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.270684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1485f25-1293-4ac2-bc28-8dc3ed38a690 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.271725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1485f25-1293-4ac2-bc28-8dc3ed38a690 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.274338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1485f25-1293-4ac2-bc28-8dc3ed38a690 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.292442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 921/1854] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:40 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:40.296392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.303860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcfe1cb2-4a24-4de8-bbf7-a0c6aaacc176 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.310267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.310267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.310267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.310905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] PATH_INFO: `/v3/projects/cde9c0cb2b484914a0635dabc728aeb7/users/dd705ed7652f4ec2b11575538fdaf6b5/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.328189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cde9c0cb2b484914a0635dabc728aeb7, user_id=dd705ed7652f4ec2b11575538fdaf6b5, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.330927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.423839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 934/1855] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:40 2022] PUT /identity/v3/projects/cde9c0cb2b484914a0635dabc728aeb7/users/dd705ed7652f4ec2b11575538fdaf6b5/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:40.424639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.424931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.425210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.472774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] MFA Rules not processed for user `dd705ed7652f4ec2b11575538fdaf6b5`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:40.474224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:40.565905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.574462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 922/1856] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:40 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 147 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:40.580905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.583824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:40.643076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d6ca96d-c18a-4e69-9781-ba9ce018d3f7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.649413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.650639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.650928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.651216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.652632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.655171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.688690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec6359c-4761-4ba1-9b1b-efab17f6028f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.697661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.698882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.699254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.699757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] PATH_INFO: `/v3/users/cf00e855ea244a93ad658a5dc3c865a1` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.708528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] RBAC: Authorizing `identity:delete_user(user_id=cf00e855ea244a93ad658a5dc3c865a1)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.710568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.795646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 935/1857] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:40 2022] POST /identity/v3/projects => generated 387 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:40.799199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.870960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b4f1ee-bdb6-43bd-bbf7-5e2f53e636e4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.877249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.877917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.878209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.878492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.879260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.881288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:40.950404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'cf00e855ea244a93ad658a5dc3c865a1'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:40.950995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'cf00e855ea244a93ad658a5dc3c865a1'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:40.965590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 936/1858] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:40 2022] POST /identity/v3/users => generated 300 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:40.966423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:40.968058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:40.979940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f182159d-ed4a-4189-9ca4-84cb451e625e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:40.985013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:40.985872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:40.986153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:40.986455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] PATH_INFO: `/v3/projects/9cb701214ced4b65b02d69435f9f3c27` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:40.987789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9cb701214ced4b65b02d69435f9f3c27)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:40.992109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.079603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'cf00e855ea244a93ad658a5dc3c865a1'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.090293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'cf00e855ea244a93ad658a5dc3c865a1'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.100811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'cf00e855ea244a93ad658a5dc3c865a1'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.107600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 923/1859] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:40 2022] DELETE /identity/v3/users/cf00e855ea244a93ad658a5dc3c865a1 => generated 0 bytes in 527 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:41.108492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.109859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.112467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:41.138785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9cb701214ced4b65b02d69435f9f3c27', 'request_id': 'req-10483c6a-5a66-4ca5-a046-de8796406ca9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.145815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9cb701214ced4b65b02d69435f9f3c27', 'request_id': 'req-10483c6a-5a66-4ca5-a046-de8796406ca9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.155578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9cb701214ced4b65b02d69435f9f3c27', 'request_id': 'req-10483c6a-5a66-4ca5-a046-de8796406ca9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.176748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b334063-0fb3-4f4c-afa2-325773c41475 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.233331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.234131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] The token cache is being invalidate because project 9cb701214ced4b65b02d69435f9f3c27 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:41.234431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.234808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.235085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-10483c6a-5a66-4ca5-a046-de8796406ca9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.235231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.235395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.236146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.239775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 937/1860] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:40 2022] DELETE /identity/v3/projects/9cb701214ced4b65b02d69435f9f3c27 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:41.240054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.240148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.240148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.240148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.256671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:41.257894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.261959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 924/1861] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:41.263581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.265772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.266035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 938/1862] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:41.266805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.268238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.280425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-343eed59-b81f-4fb5-ae39-e270e5785607 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.280715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-890bd195-f813-499d-9cc2-73bf7d6d453e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.285173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.285345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.286149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.286271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.286369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.286501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.286677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] PATH_INFO: `/v3/projects/a6baf8e28e824c91a4ddab39d13230de/users/15274d70089b403787ce02fd1d432445/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.286849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] PATH_INFO: `/v3/users/6a4f77f9526349d1a650ba40141e6baf` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.288121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a4f77f9526349d1a650ba40141e6baf)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.290049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.303906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6baf8e28e824c91a4ddab39d13230de, user_id=15274d70089b403787ce02fd1d432445, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.310435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.374593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 939/1863] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:41 2022] PUT /identity/v3/projects/a6baf8e28e824c91a4ddab39d13230de/users/15274d70089b403787ce02fd1d432445/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:41.375419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.377072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.432078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-994df39d-a5e8-4fdb-9894-096580e12687 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.436047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.436734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.437014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.437300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.474497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a4f77f9526349d1a650ba40141e6baf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.475164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a4f77f9526349d1a650ba40141e6baf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.503580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.505083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.514926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 940/1864] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:41.515836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.524946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56954a22-129d-428b-bd75-f6fa4b81092f service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.532118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.532829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.533116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.533394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.534114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.535698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.546695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 941/1865] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:41 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:41.550037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.566476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a4f77f9526349d1a650ba40141e6baf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.566663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b32ca9-7c21-440d-a14b-4f7a20610c67 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.566762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a4f77f9526349d1a650ba40141e6baf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.567780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d14cc8d-a1a0-4af2-8b0f-17c48160d562 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.568742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cc8d-a1a0-4af2-8b0f-17c48160d562 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.569035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cc8d-a1a0-4af2-8b0f-17c48160d562 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.569370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d14cc8d-a1a0-4af2-8b0f-17c48160d562 admin admin] PATH_INFO: `/v3/projects/a6baf8e28e824c91a4ddab39d13230de/users/15274d70089b403787ce02fd1d432445/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.571496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14cc8d-a1a0-4af2-8b0f-17c48160d562 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6baf8e28e824c91a4ddab39d13230de, user_id=15274d70089b403787ce02fd1d432445, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.574392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d14cc8d-a1a0-4af2-8b0f-17c48160d562 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.585687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a4f77f9526349d1a650ba40141e6baf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:41.595988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 942/1866] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:41 2022] PUT /identity/v3/projects/a6baf8e28e824c91a4ddab39d13230de/users/15274d70089b403787ce02fd1d432445/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:41.612032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.612032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.612032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.612311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 925/1867] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:41 2022] DELETE /identity/v3/users/6a4f77f9526349d1a650ba40141e6baf => generated 0 bytes in 341 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:41.627116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] MFA Rules not processed for user `15274d70089b403787ce02fd1d432445`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:41.628363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.657372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:41.703695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 943/1868] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:41 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:41.710909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:41.712550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:41.772918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067835a9-44ac-457e-baaf-0d8012534cbf admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:41.776636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:41.777288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:41.777573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:41.777837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:41.787975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:41.789477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:41.800288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 926/1869] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:41 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:42.061549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:42.064398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:42.162065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d7717bd-efc2-4b04-8343-4c8337930e7e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:42.168199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:42.169104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.169415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.169939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] PATH_INFO: `/v3/projects/c637612376914a3ba9c0554e04072232` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.171138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] RBAC: Authorizing `identity:delete_project(project_id=c637612376914a3ba9c0554e04072232)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:42.173088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:42.253625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:42.255269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:42.297371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e61d925d-4536-499d-8e0d-4663686e0cfb service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:42.317067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c637612376914a3ba9c0554e04072232', 'request_id': 'req-442a1a38-37e2-4abe-a5c3-8742194cdd6f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.340766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c637612376914a3ba9c0554e04072232', 'request_id': 'req-442a1a38-37e2-4abe-a5c3-8742194cdd6f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.353368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c637612376914a3ba9c0554e04072232', 'request_id': 'req-442a1a38-37e2-4abe-a5c3-8742194cdd6f'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.369146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:42.370333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.370526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.370751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] PATH_INFO: `/v3/projects/2d031ae91449428aa7b26b44cc190745` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.372854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2d031ae91449428aa7b26b44cc190745)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:42.374884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:42.447652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] The token cache is being invalidate because project c637612376914a3ba9c0554e04072232 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:42.447763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.448671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.454158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 944/1870] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:42 2022] DELETE /identity/v3/projects/c637612376914a3ba9c0554e04072232 => generated 0 bytes in 390 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:42.455040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:42.456603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:42.474294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442a1a38-37e2-4abe-a5c3-8742194cdd6f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:42.525011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2d031ae91449428aa7b26b44cc190745', 'request_id': 'req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.534636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2d031ae91449428aa7b26b44cc190745', 'request_id': 'req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.554561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:42.555446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:42.556389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.556675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.556949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] PATH_INFO: `/v3/users/152bf6acced24405a93b40377e78f883` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.558172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=152bf6acced24405a93b40377e78f883)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:42.560194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:42.631898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2d031ae91449428aa7b26b44cc190745', 'request_id': 'req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.645047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] The token cache is being invalidate because project 2d031ae91449428aa7b26b44cc190745 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:42.645342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.646015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-a0fa0bd3-925e-4c3e-9439-670e34b0c6d6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.647781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 927/1871] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:42 2022] DELETE /identity/v3/projects/2d031ae91449428aa7b26b44cc190745 => generated 0 bytes in 395 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:42.653781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a8383-9fca-475a-9399-877e9bd352e5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.654090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a8383-9fca-475a-9399-877e9bd352e5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.654376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b00a8383-9fca-475a-9399-877e9bd352e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.683384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b00a8383-9fca-475a-9399-877e9bd352e5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:42.689979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b00a8383-9fca-475a-9399-877e9bd352e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:42.699507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 928/1872] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 47 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:42.707549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.707837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.708175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.725972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:42.727155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:42.734985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 929/1873] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:42.739075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:42.748312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b8ff244-be45-4143-abc4-fe419d52f76a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:42.752983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:42.753604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.753887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.754160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.754951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:42.755256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:42.756914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:42.770173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 930/1874] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:56:42.773849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:42.783481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f7b567e-b9c9-4f27-b7d9-2b51c6ea66e4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:42.788173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:42.788871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:42.789163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:42.789441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:42.790115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:42.792014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:42.811059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '152bf6acced24405a93b40377e78f883'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.811990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '152bf6acced24405a93b40377e78f883'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.929761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 931/1875] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:42 2022] POST /identity/v3/projects => generated 377 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:42.934971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:42.948712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '152bf6acced24405a93b40377e78f883'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.960049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '152bf6acced24405a93b40377e78f883'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.973996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '152bf6acced24405a93b40377e78f883'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:42.987037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 945/1876] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:42 2022] DELETE /identity/v3/users/152bf6acced24405a93b40377e78f883 => generated 0 bytes in 529 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:43.026889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b79c2f3-2c92-4b24-a3c4-82e8c62ed6f1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:43.042721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:43.116648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:43.116648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:43.116648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:43.116648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:43.117119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:43.117119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.031133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 932/1877] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:42 2022] POST /identity/v3/users => generated 289 bytes in 1094 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:44.032047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.033936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.044626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.045648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.049274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25937327-dd7f-4aaf-9a6b-4beb143809c9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.055502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.057499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.058077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.060302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.060302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.061337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84556004-95a1-4510-bdbd-adce3471ba72 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.061620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.066866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.069209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.069503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.069791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] PATH_INFO: `/v3/users/dd705ed7652f4ec2b11575538fdaf6b5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.071331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd705ed7652f4ec2b11575538fdaf6b5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.073478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.097069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 946/1878] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:44.100665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.112792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c961862b-1232-47cf-82ec-c30bd91f26c0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.117804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.118689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.118972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.119453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.120571 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.121818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.132510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 947/1879] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:44 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:44.136785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.148691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90aebe61-6e5c-453f-b3ed-5c1bd819767b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.154311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d778c94a-320f-4381-b484-a3ab860e9475 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.155256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778c94a-320f-4381-b484-a3ab860e9475 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.155562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778c94a-320f-4381-b484-a3ab860e9475 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.155803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d778c94a-320f-4381-b484-a3ab860e9475 admin admin] PATH_INFO: `/v3/projects/6c61cc884ce540c6afc8d4723e04299f/users/82e0f549da3e4ed9a7fe4f488c652b1b/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.173008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778c94a-320f-4381-b484-a3ab860e9475 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6c61cc884ce540c6afc8d4723e04299f, user_id=82e0f549da3e4ed9a7fe4f488c652b1b, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.175996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d778c94a-320f-4381-b484-a3ab860e9475 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.243469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 948/1880] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:44 2022] PUT /identity/v3/projects/6c61cc884ce540c6afc8d4723e04299f/users/82e0f549da3e4ed9a7fe4f488c652b1b/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:44.255271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.255782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.256110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.295744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] MFA Rules not processed for user `82e0f549da3e4ed9a7fe4f488c652b1b`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:44.295744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.298211 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd705ed7652f4ec2b11575538fdaf6b5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.298923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd705ed7652f4ec2b11575538fdaf6b5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.359518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 949/1881] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:44 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:44.370374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.374924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd705ed7652f4ec2b11575538fdaf6b5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.385213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd705ed7652f4ec2b11575538fdaf6b5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.401117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd705ed7652f4ec2b11575538fdaf6b5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.410903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 933/1882] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:44 2022] DELETE /identity/v3/users/dd705ed7652f4ec2b11575538fdaf6b5 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:44.412124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.413720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.469739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37b2992c-bf9c-49f9-a344-53fe5f6ed06b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.525164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27da3904-0344-4a3d-9099-63475df2582b admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.531996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.532888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.533200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.533499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] PATH_INFO: `/v3/projects/c3ad7d438b2c45daa0d4ebef78702667` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.534692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c3ad7d438b2c45daa0d4ebef78702667)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.536722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.544319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c75fc399-2195-4dcf-88f7-b9184063f21a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.549546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fc399-2195-4dcf-88f7-b9184063f21a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.549546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fc399-2195-4dcf-88f7-b9184063f21a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.549546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c75fc399-2195-4dcf-88f7-b9184063f21a admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.549546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fc399-2195-4dcf-88f7-b9184063f21a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.549546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c75fc399-2195-4dcf-88f7-b9184063f21a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.657208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c3ad7d438b2c45daa0d4ebef78702667', 'request_id': 'req-9b018ba9-4e51-4467-8e05-99520cad2e7a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.662763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c3ad7d438b2c45daa0d4ebef78702667', 'request_id': 'req-9b018ba9-4e51-4467-8e05-99520cad2e7a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.712341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 950/1883] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:44 2022] POST /identity/v3/projects => generated 375 bytes in 342 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:44.714597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.714917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.715189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.741917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:44.743827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.808099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 951/1884] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:44.809105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.811969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.816447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c3ad7d438b2c45daa0d4ebef78702667', 'request_id': 'req-9b018ba9-4e51-4467-8e05-99520cad2e7a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.826145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecd8216-2309-468a-b40d-dfc90532b1d9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.833208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.834360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.834793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.835247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.836043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.837909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] The token cache is being invalidate because project c3ad7d438b2c45daa0d4ebef78702667 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:44.838415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.838903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:44.839362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:44.841891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 934/1885] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:44 2022] DELETE /identity/v3/projects/c3ad7d438b2c45daa0d4ebef78702667 => generated 0 bytes in 430 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:44.844097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:44.845771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.856460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:44.924628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b018ba9-4e51-4467-8e05-99520cad2e7a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:44.973670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:44.974245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:44.974531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:44.974811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:44.976664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:44.988490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:44.989915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c007af7a-4a35-4c1c-9bd2-46529cbc5c9e service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.002816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 935/1886] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:45.002816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969706-e687-4e26-903a-bdc9a545a02a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.002816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969706-e687-4e26-903a-bdc9a545a02a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.002816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94969706-e687-4e26-903a-bdc9a545a02a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.018593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 952/1887] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:44 2022] POST /identity/v3/users => generated 294 bytes in 210 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:45.021121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.025398 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-94969706-e687-4e26-903a-bdc9a545a02a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:45.026372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.026601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-94969706-e687-4e26-903a-bdc9a545a02a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.046596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 936/1888] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:45.055820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.056238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.056519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.060688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519574d7-66b2-46d9-9bfa-1890903cf5f4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.078472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.079063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.079442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.079628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.081000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:45.081124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.082661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.082808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.103808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 937/1889] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 48 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:45.107443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.112118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 953/1890] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:45.113561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.120900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57b54126-9a6f-4e6e-aa88-0a22c728a013 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.127140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.127924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.128286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.128636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.129878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:45.130075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cfdff85-425c-443a-b0db-906382ecf1fb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.130498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.132207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.136171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.138517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.138517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.138517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] PATH_INFO: `/v3/projects/19a66ac4cbf84fd586a59bc8eaa759ca/users/c6c188648fbd4af0b980a717c1a3d6e2/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.153160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 938/1891] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:56:45.154943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.161043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a66ac4cbf84fd586a59bc8eaa759ca, user_id=c6c188648fbd4af0b980a717c1a3d6e2, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.165086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.170003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d75a34-adba-44e6-9cc0-51b126d0c9f7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.174873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.175545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.175829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.176139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.176836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.178760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.261665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 954/1892] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:45 2022] PUT /identity/v3/projects/19a66ac4cbf84fd586a59bc8eaa759ca/users/c6c188648fbd4af0b980a717c1a3d6e2/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 145 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:45.263343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.327018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 939/1893] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:45 2022] POST /identity/v3/projects => generated 409 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:45.328281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.351998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2097306c-2488-4ed9-96b9-8e8a87a67080 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.442494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1221a3f-3b5b-4345-9da9-71db0023c9ba admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.448038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.448729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.449017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.449295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.450716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.453095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.469384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.470391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.470698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.471003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.473389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.473389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.489468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 955/1894] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:45.489468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.502324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda61410-6f11-439d-979c-2c10997923ed admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.507841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b92b2b-82c2-42e3-9b11-a19513af0839 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.508217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b92b2b-82c2-42e3-9b11-a19513af0839 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.508502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b92b2b-82c2-42e3-9b11-a19513af0839 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.508912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b92b2b-82c2-42e3-9b11-a19513af0839 admin admin] PATH_INFO: `/v3/projects/19a66ac4cbf84fd586a59bc8eaa759ca/users/c6c188648fbd4af0b980a717c1a3d6e2/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.511005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b92b2b-82c2-42e3-9b11-a19513af0839 admin admin] RBAC: Authorizing `identity:create_grant(project_id=19a66ac4cbf84fd586a59bc8eaa759ca, user_id=c6c188648fbd4af0b980a717c1a3d6e2, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.514606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b92b2b-82c2-42e3-9b11-a19513af0839 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.535701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 956/1895] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:45 2022] PUT /identity/v3/projects/19a66ac4cbf84fd586a59bc8eaa759ca/users/c6c188648fbd4af0b980a717c1a3d6e2/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:45.547396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 940/1896] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:45 2022] POST /identity/v3/users => generated 305 bytes in 217 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:45.547604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.547604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.547604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.548528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.563126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8549859-a26b-4a9d-bf52-1d89ccb3b833 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.596172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] MFA Rules not processed for user `c6c188648fbd4af0b980a717c1a3d6e2`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:45.598438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.649647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.650378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.650661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.650935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.651673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.653323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.666149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 941/1897] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:45.669352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.670598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.683764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb994b1f-5a85-43df-8112-2f87512bbda1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.697033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.698029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.698380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.699260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] PATH_INFO: `/v3/projects/cde9c0cb2b484914a0635dabc728aeb7` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.703473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] RBAC: Authorizing `identity:delete_project(project_id=cde9c0cb2b484914a0635dabc728aeb7)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.705665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.708885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 957/1898] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:45 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:45.709805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.724641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89898410-51f7-40d2-8750-21dd19b71fd0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.730101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.730901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.732406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.732406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.732406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.734243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.749010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 958/1899] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:45 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:45.749010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.749010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.782406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:45.813636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc1d75d9-a251-45f2-bdce-902e5c494dad admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.818577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.819696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.820072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.820574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:45.822673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.830766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cde9c0cb2b484914a0635dabc728aeb7', 'request_id': 'req-dac6cfd8-82ad-4e37-b624-6d404f41b88c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:45.836811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cde9c0cb2b484914a0635dabc728aeb7', 'request_id': 'req-dac6cfd8-82ad-4e37-b624-6d404f41b88c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:45.884072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:45.885664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:45.896284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 959/1900] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:45 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 150 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:45.897338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.908525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02b88a9-5d3c-46ce-81f5-d7be7d06d6e1 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:45.918344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cde9c0cb2b484914a0635dabc728aeb7', 'request_id': 'req-dac6cfd8-82ad-4e37-b624-6d404f41b88c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:45.936478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] The token cache is being invalidate because project cde9c0cb2b484914a0635dabc728aeb7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:45.936981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:45.937851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:45.944118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 942/1901] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:45 2022] DELETE /identity/v3/projects/cde9c0cb2b484914a0635dabc728aeb7 => generated 0 bytes in 273 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:45.945433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:45.947216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:45.994585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b0c7055-54ea-43ac-9a5b-32b5f59f0a88 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:45.995724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7055-54ea-43ac-9a5b-32b5f59f0a88 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:45.996768 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7055-54ea-43ac-9a5b-32b5f59f0a88 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:45.997608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b0c7055-54ea-43ac-9a5b-32b5f59f0a88 admin admin] PATH_INFO: `/v3/projects/80a65c7474fc4e1f830606fddd3496df/users/12fc5277226e4c50bc328aa35a2a12ff/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.023149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c7055-54ea-43ac-9a5b-32b5f59f0a88 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80a65c7474fc4e1f830606fddd3496df, user_id=12fc5277226e4c50bc328aa35a2a12ff, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:46.026566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b0c7055-54ea-43ac-9a5b-32b5f59f0a88 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:46.053926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac6cfd8-82ad-4e37-b624-6d404f41b88c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:46.061617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:46.062674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.066083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.066083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] PATH_INFO: `/v3/users/15274d70089b403787ce02fd1d432445` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.066283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=15274d70089b403787ce02fd1d432445)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:46.071150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:46.115071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:46.123844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 960/1902] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:45 2022] PUT /identity/v3/projects/80a65c7474fc4e1f830606fddd3496df/users/12fc5277226e4c50bc328aa35a2a12ff/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 227 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:46.137930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8947e-87fb-4776-8659-753d4c18c53c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.143793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8947e-87fb-4776-8659-753d4c18c53c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.146282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c8947e-87fb-4776-8659-753d4c18c53c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.192067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-55c8947e-87fb-4776-8659-753d4c18c53c None None] MFA Rules not processed for user `12fc5277226e4c50bc328aa35a2a12ff`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:46.193490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-55c8947e-87fb-4776-8659-753d4c18c53c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:46.297613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 961/1903] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:46.301112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0365f-a84b-40b1-ae33-1f1c8137d9cf None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.301831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0365f-a84b-40b1-ae33-1f1c8137d9cf None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.302291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb0365f-a84b-40b1-ae33-1f1c8137d9cf None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.304088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 962/1904] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:56:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:56:46.305485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0365f-a84b-40b1-ae33-1f1c8137d9cf None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:46.308150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bfb0365f-a84b-40b1-ae33-1f1c8137d9cf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:46.351797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '15274d70089b403787ce02fd1d432445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:46.352424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '15274d70089b403787ce02fd1d432445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:46.408667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb0365f-a84b-40b1-ae33-1f1c8137d9cf None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:46.413592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:46.414360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.414710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.415055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.415852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:46.417898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:46.434302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '15274d70089b403787ce02fd1d432445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:46.444878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '15274d70089b403787ce02fd1d432445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:46.461735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0138d7f0-dfe8-49c9-b18e-29e3b1f0a9f6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '15274d70089b403787ce02fd1d432445'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:46.471396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 943/1905] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:45 2022] DELETE /identity/v3/users/15274d70089b403787ce02fd1d432445 => generated 0 bytes in 526 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:46.474313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.475263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.475737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.505331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 963/1906] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:46 2022] POST /identity/v3/projects => generated 409 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:46.509400 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:46.510309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:46.511207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:46.514136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:46.665302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 944/1907] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:56:46 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:46.668380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314bc662-c122-49ae-8817-4ea8b45fd4fd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:46.681271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:46.682825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:46.683714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:46.685420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.688424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.688876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.689739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:46.694267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:46.769809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d7cee4a-a08a-4ab1-aef3-3508603718f9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:46.787630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:46.792677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.793163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.795815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.818030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:46.821151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:46.838860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 964/1908] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:46 2022] POST /identity/v3/users => generated 311 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:46.842476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:46.846220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 945/1909] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:46 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 166 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:46.862639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a516022f-e364-466e-8238-902d2cd48430 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:46.882443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:46.886085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:46.886085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:46.886276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:46.887434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:46.890580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:46.924010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:46.924164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 965/1910] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:46 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:46.941782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e937d9-3776-4251-a742-8e860c2d0f20 service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.001870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.003116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.003116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.003294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] PATH_INFO: `/v3/projects/d34b9a1c989948b08d3bf5a80c224892/users/399026bfda314e10a603382d0a533908/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.039364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34b9a1c989948b08d3bf5a80c224892, user_id=399026bfda314e10a603382d0a533908, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.040927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:47.190783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 946/1911] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:46 2022] PUT /identity/v3/projects/d34b9a1c989948b08d3bf5a80c224892/users/399026bfda314e10a603382d0a533908/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:47.194629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:47.212930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1962bc96-44c3-4035-ba87-a1e49d018d80 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.314693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.316191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.316686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.317165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.318132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.320040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:47.333125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 966/1912] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:47 2022] GET /identity/v3/roles => generated 1420 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:47.337062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:47.355902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:47.359036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:47.361467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7337e66-874d-4dba-a519-2c63ab1f93ee admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.380505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5de975-1fb4-49a1-971d-f9cc9d3b8e3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.381618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de975-1fb4-49a1-971d-f9cc9d3b8e3b admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.381969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de975-1fb4-49a1-971d-f9cc9d3b8e3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.382500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5de975-1fb4-49a1-971d-f9cc9d3b8e3b admin admin] PATH_INFO: `/v3/projects/d34b9a1c989948b08d3bf5a80c224892/users/399026bfda314e10a603382d0a533908/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.389601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5de975-1fb4-49a1-971d-f9cc9d3b8e3b admin admin] RBAC: Authorizing `identity:create_grant(project_id=d34b9a1c989948b08d3bf5a80c224892, user_id=399026bfda314e10a603382d0a533908, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.393322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5de975-1fb4-49a1-971d-f9cc9d3b8e3b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:47.398460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b732e63-3253-4aa9-aef5-c2cb1f46e3bc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.408888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.419258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 947/1913] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:47 2022] PUT /identity/v3/projects/d34b9a1c989948b08d3bf5a80c224892/users/399026bfda314e10a603382d0a533908/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:47.423748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.423748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.423748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] PATH_INFO: `/v3/projects/a6baf8e28e824c91a4ddab39d13230de` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.424375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.424375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.424375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.426347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6baf8e28e824c91a4ddab39d13230de)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.428188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:47.470259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] MFA Rules not processed for user `399026bfda314e10a603382d0a533908`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:47.471776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:47.590262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 948/1914] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:47.603005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:47.604580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:47.635421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6baf8e28e824c91a4ddab39d13230de', 'request_id': 'req-a27a8e16-47e1-4169-a6fd-02f9847c6983'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:47.646224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6baf8e28e824c91a4ddab39d13230de', 'request_id': 'req-a27a8e16-47e1-4169-a6fd-02f9847c6983'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:47.657158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6baf8e28e824c91a4ddab39d13230de', 'request_id': 'req-a27a8e16-47e1-4169-a6fd-02f9847c6983'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:47.677310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9903a8df-c866-4b49-b36a-709c3b91822c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.726658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a158f252-2b18-4769-a49d-1f2b1e83f8ba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.727323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f252-2b18-4769-a49d-1f2b1e83f8ba service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.727608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f252-2b18-4769-a49d-1f2b1e83f8ba service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.728083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a158f252-2b18-4769-a49d-1f2b1e83f8ba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.732872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] The token cache is being invalidate because project a6baf8e28e824c91a4ddab39d13230de was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:47.733918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:47.733918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a27a8e16-47e1-4169-a6fd-02f9847c6983 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:47.735916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 967/1915] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:47 2022] DELETE /identity/v3/projects/a6baf8e28e824c91a4ddab39d13230de => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:47.741182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.741532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.741752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.760722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:47.761928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:47.798220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158f252-2b18-4769-a49d-1f2b1e83f8ba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.799674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a158f252-2b18-4769-a49d-1f2b1e83f8ba service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:47.809428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 949/1916] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:47 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:47.832019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 968/1917] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:47.845570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.845570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.845570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.860657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:47.861944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:47.872506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 950/1918] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:47.876782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:47.891792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-531a21e9-3a7c-413c-a3d4-f5e1554001f9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.899256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.900308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.901073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.901371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.902396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:47.902715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.905037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:47.921418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 969/1919] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 45 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:47.928712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:47.939005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c39395bd-9723-4e4c-88c5-894bca6858ca None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:47.944114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:47.944878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:47.945233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:47.945626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:47.946389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:47.948640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.059051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 951/1920] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:47 2022] POST /identity/v3/projects => generated 375 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:48.060909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.071202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0be47f04-d0a3-4ba3-93b3-3b51bef720cd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:48.135148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:48.135829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.136197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.136480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.137166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:48.139727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.197857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=3,keystone:SELECT=96,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:48.255239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 970/1921] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:48 2022] POST /identity/v3/users => generated 288 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:48.259978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.283862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36512dbc-c16e-48bb-ba0b-febe0b5f5a99 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:48.288776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:48.289455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.289737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.290009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.290742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:48.292385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.300448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.302340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:48.303906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 952/1922] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:48.307040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.317075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c3f15aa-0d96-46d9-9d61-67c81b6ddbc6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:48.321674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:48.322315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.322597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.322872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.323585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:48.325256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.334930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 953/1923] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:48 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:48.338929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.350436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10bdadb9-6b30-485b-9e11-c4e806837cc2 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:48.355762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5695a1c0-a5e3-47f2-bd4f-b1fa588fa6e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:48.357260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a1c0-a5e3-47f2-bd4f-b1fa588fa6e7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.357544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a1c0-a5e3-47f2-bd4f-b1fa588fa6e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.357818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5695a1c0-a5e3-47f2-bd4f-b1fa588fa6e7 admin admin] PATH_INFO: `/v3/projects/a23c543d8b4341e6a0d9d3e7bad8acd1/users/95496ebf94b046ff8d64c27ed2243e28/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.370299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8d4fec6-bd6a-4708-80bb-c42876c8bf7b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:48.384930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:48.384930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.384930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.384930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.385293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695a1c0-a5e3-47f2-bd4f-b1fa588fa6e7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a23c543d8b4341e6a0d9d3e7bad8acd1, user_id=95496ebf94b046ff8d64c27ed2243e28, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:48.387315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:48.387468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5695a1c0-a5e3-47f2-bd4f-b1fa588fa6e7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.472484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 954/1924] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:48 2022] PUT /identity/v3/projects/a23c543d8b4341e6a0d9d3e7bad8acd1/users/95496ebf94b046ff8d64c27ed2243e28/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:48.477802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:48.477888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.477888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.478016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.478102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.487477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 971/1925] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:48 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 187 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:48.516212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] MFA Rules not processed for user `95496ebf94b046ff8d64c27ed2243e28`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:48.519563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:48.658831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:48.680297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 955/1926] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 205 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:48.685129 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.700508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9d5805c-07e9-473e-b104-e98e40e0bee8 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:48.802842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:48.803732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:48.804387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:48.804671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:48.805668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:48.807958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:48.913149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.914831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:48.925535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 972/1927] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:48 2022] POST /identity/v3/projects => generated 375 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:48.929428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:48.935838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404f8a16-ea8f-4ba8-9382-cb00e4186439 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.000024 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c3584b-1021-4cc1-9737-e6f3ead55fb6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.010911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.011817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.012234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.012920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] PATH_INFO: `/v3/users/6b7b25f9563745cd832aa68d253ad80d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.014140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6b7b25f9563745cd832aa68d253ad80d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.015561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.016104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.016279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.016575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.016840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.017541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.019563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.124726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 973/1928] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:48 2022] POST /identity/v3/users => generated 288 bytes in 194 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:49.127331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.136883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99737333-0790-48e0-80d3-4a0942019b7a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.141945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.142581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.142868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.143149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.143890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.145946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.156834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 974/1929] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:49.160497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.169737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfcbea42-8267-4a02-91c2-559396bd8cd4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.174291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.174892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.175176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.175451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.176172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.177685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.187156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 975/1930] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 27 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:56:49.190791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.200483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64de6c14-2b85-454a-8455-b2d1837ce078 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.205437 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc8259b0-1946-4434-bd5c-3ffe523c2e39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.206302 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8259b0-1946-4434-bd5c-3ffe523c2e39 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.216769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8259b0-1946-4434-bd5c-3ffe523c2e39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.216769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc8259b0-1946-4434-bd5c-3ffe523c2e39 admin admin] PATH_INFO: `/v3/projects/64040e0cca9346cbb2d47ef165ef62c3/users/bb063aa068844f12899649f23ab79962/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.223523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8259b0-1946-4434-bd5c-3ffe523c2e39 admin admin] RBAC: Authorizing `identity:create_grant(project_id=64040e0cca9346cbb2d47ef165ef62c3, user_id=bb063aa068844f12899649f23ab79962, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.226209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc8259b0-1946-4434-bd5c-3ffe523c2e39 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.250045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6b7b25f9563745cd832aa68d253ad80d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:49.250769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6b7b25f9563745cd832aa68d253ad80d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:49.301001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 976/1931] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:49 2022] PUT /identity/v3/projects/64040e0cca9346cbb2d47ef165ef62c3/users/bb063aa068844f12899649f23ab79962/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:49.310889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57015c36-c45d-439a-b832-d01f551e376a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.311182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57015c36-c45d-439a-b832-d01f551e376a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.311471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57015c36-c45d-439a-b832-d01f551e376a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.317150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6b7b25f9563745cd832aa68d253ad80d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:49.325309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6b7b25f9563745cd832aa68d253ad80d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:49.340035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28d1a5c1-4950-4b8b-96b8-7f783aa16998 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6b7b25f9563745cd832aa68d253ad80d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:49.350468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 956/1932] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:48 2022] DELETE /identity/v3/users/6b7b25f9563745cd832aa68d253ad80d => generated 0 bytes in 435 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:49.351803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-57015c36-c45d-439a-b832-d01f551e376a None None] MFA Rules not processed for user `bb063aa068844f12899649f23ab79962`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:49.352021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.352190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.352530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.353191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-57015c36-c45d-439a-b832-d01f551e376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:49.378142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:49.378795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:49.428351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 977/1933] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:49 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:49.433487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57015c36-c45d-439a-b832-d01f551e376a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.437027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-57015c36-c45d-439a-b832-d01f551e376a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:49.452124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 957/1934] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:49.461407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.463007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:49.511240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57015c36-c45d-439a-b832-d01f551e376a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.516733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.517407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.517698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.517980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.518677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.520629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.523692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa1a6a3-565b-49c0-9ae6-46f23589df96 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.527565 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.528266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.528547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.528825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.538338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.539800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.549793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 958/1935] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:49.618993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 978/1936] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:49 2022] POST /identity/v3/projects => generated 377 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:49.620183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.630416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6922d37-9393-4842-889f-ba82acd056d2 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.694076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.694789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.695496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.695779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.696527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.698642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.862101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 959/1937] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:49 2022] POST /identity/v3/users => generated 295 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:49.869008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.880835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3271e596-f64d-4d00-bb88-c6acb24a4548 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.886482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.887798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.888191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.893235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.893235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.893235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:49.905798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 979/1938] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:49 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:49.909891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:49.920560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3eedf051-2948-4671-979e-475ab6d69a9c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:49.926124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:49.927111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:49.927821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:49.927821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] PATH_INFO: `/v3/projects/3cc2f370bae44d43a1996900eb77a703/users/24caffffcb1e4d2b800e06b7a7a5d55a/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:49.952444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc2f370bae44d43a1996900eb77a703, user_id=24caffffcb1e4d2b800e06b7a7a5d55a, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:49.955681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:50.068839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 960/1939] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:49 2022] PUT /identity/v3/projects/3cc2f370bae44d43a1996900eb77a703/users/24caffffcb1e4d2b800e06b7a7a5d55a/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:50.070828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:50.072197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:50.075532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:50.110562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7cb0212-5822-4ec9-88b1-8dac2e96e07c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:50.142443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e177ec6-6bd9-4d68-9990-95aea8a8264e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:50.148299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:50.148980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:50.149269 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:50.149553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:50.150356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:50.151911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:50.162815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 961/1940] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:50 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:50.166818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:50.170556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:50.171463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:50.171747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:50.172053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] PATH_INFO: `/v3/projects/fdb6d472385f4d198dc532b006fd711b` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:50.173231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdb6d472385f4d198dc532b006fd711b)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:50.175141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:50.176546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1e6f839-8de5-4f13-82c9-ef978e45c722 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:50.181152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6594fe6-86e9-48ef-be4b-bd88908cdfec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:50.182019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6594fe6-86e9-48ef-be4b-bd88908cdfec admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:50.182304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6594fe6-86e9-48ef-be4b-bd88908cdfec admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:50.182583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6594fe6-86e9-48ef-be4b-bd88908cdfec admin admin] PATH_INFO: `/v3/projects/3cc2f370bae44d43a1996900eb77a703/users/24caffffcb1e4d2b800e06b7a7a5d55a/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:50.185214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6594fe6-86e9-48ef-be4b-bd88908cdfec admin admin] RBAC: Authorizing `identity:create_grant(project_id=3cc2f370bae44d43a1996900eb77a703, user_id=24caffffcb1e4d2b800e06b7a7a5d55a, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:50.187910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6594fe6-86e9-48ef-be4b-bd88908cdfec admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:50.198114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 962/1941] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:50 2022] PUT /identity/v3/projects/3cc2f370bae44d43a1996900eb77a703/users/24caffffcb1e4d2b800e06b7a7a5d55a/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:50.208522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:50.208823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:50.209107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:50.234699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] MFA Rules not processed for user `24caffffcb1e4d2b800e06b7a7a5d55a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:50.235929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:50.260380 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdb6d472385f4d198dc532b006fd711b', 'request_id': 'req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.267093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdb6d472385f4d198dc532b006fd711b', 'request_id': 'req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.272864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdb6d472385f4d198dc532b006fd711b', 'request_id': 'req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.303196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 963/1942] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:50 2022] POST /identity/v3/auth/tokens => generated 3311 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:50.315137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:50.316547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:50.376013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84433787-9857-4b29-ac45-dfba4d58a10c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:50.404935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] The token cache is being invalidate because project fdb6d472385f4d198dc532b006fd711b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:50.405232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.405878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.407669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 980/1943] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:50 2022] DELETE /identity/v3/projects/fdb6d472385f4d198dc532b006fd711b => generated 0 bytes in 337 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:50.411507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:50.412837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:50.424123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=87,keystone:INSERT=6,keystone:DELETE=7 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:50.427153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:50.427838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:50.428148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:50.430422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:50.430422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:50.479008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-858c4d7c-983b-4ca8-8d02-cc27c3fd279e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:50.485751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:50.486638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:50.486930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:50.487214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] PATH_INFO: `/v3/users/069b71ff50c8465e879e2616ae382b8b` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:50.488467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] RBAC: Authorizing `identity:delete_user(user_id=069b71ff50c8465e879e2616ae382b8b)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:50.490505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:50.494136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:50.495577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:50.507274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 964/1944] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:50 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:50.654470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:50.667902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '069b71ff50c8465e879e2616ae382b8b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.669110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '069b71ff50c8465e879e2616ae382b8b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.740545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '069b71ff50c8465e879e2616ae382b8b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.748627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '069b71ff50c8465e879e2616ae382b8b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.760528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-11c00e68-0402-4545-b353-08e722a5e607 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '069b71ff50c8465e879e2616ae382b8b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:50.766646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 981/1945] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:50 2022] DELETE /identity/v3/users/069b71ff50c8465e879e2616ae382b8b => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:51.241305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:51.254120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65ea088-a17d-42ed-9505-c431559a1308 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:51.318468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:51.321558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:51.321558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:51.321558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] PATH_INFO: `/v3/projects/217b375272c44813ace2925ad44e0e5e` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:51.321558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=217b375272c44813ace2925ad44e0e5e)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:51.322995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:51.436502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '217b375272c44813ace2925ad44e0e5e', 'request_id': 'req-c21d82e3-6aa9-4422-b347-937a32a290d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:51.441694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '217b375272c44813ace2925ad44e0e5e', 'request_id': 'req-c21d82e3-6aa9-4422-b347-937a32a290d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:51.522601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '217b375272c44813ace2925ad44e0e5e', 'request_id': 'req-c21d82e3-6aa9-4422-b347-937a32a290d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:51.534298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] The token cache is being invalidate because project 217b375272c44813ace2925ad44e0e5e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:51.534586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:51.535203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c21d82e3-6aa9-4422-b347-937a32a290d0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:51.536758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 965/1946] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:51 2022] DELETE /identity/v3/projects/217b375272c44813ace2925ad44e0e5e => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:51.542663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:51.543146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:51.543237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:51.561607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:51.562872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:51.639399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 982/1947] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:51.647329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:51.647623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:51.647938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:51.670899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:51.675751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:51.684598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 966/1948] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:51.688751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:51.699233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364152e2-661b-4f2e-8650-f8bb0445cb47 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:51.704573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:51.705242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:51.705530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:51.705810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:51.706548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:51.707023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:51.708733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:51.721006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 983/1949] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:56:51.725780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:51.737267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91043587-29bf-4158-bd40-33f7a5b8defe None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:51.742867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:51.744264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:51.744684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:51.744970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:51.745688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:51.748362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:51.849200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 967/1950] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:51 2022] POST /identity/v3/projects => generated 379 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:56:51.850377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:51.862172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28852923-8715-4831-8ef5-1f8b79e6b671 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:51.921277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:51.921964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:51.922251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:51.922523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:51.923249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:51.925586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.012945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 984/1951] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:51 2022] POST /identity/v3/users => generated 290 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:52.016240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.026320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a7ec17d-3d6a-499b-9e45-5285d95cccff admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.031094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.031761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.032088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.032368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.033102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.034702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.045511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 968/1952] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:52.049378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.059516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6657254-360e-40f1-b4bf-7123ee664d4a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.065846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbbf7052-b00c-432e-98f3-28537770ef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.066647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7052-b00c-432e-98f3-28537770ef39 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.066998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7052-b00c-432e-98f3-28537770ef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.067280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbbf7052-b00c-432e-98f3-28537770ef39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.068042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf7052-b00c-432e-98f3-28537770ef39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.069640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbbf7052-b00c-432e-98f3-28537770ef39 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.080814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 985/1953] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:52.084966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.096637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9c197af-5392-41f9-bbeb-8aedb9892c71 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.105189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.106694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.106976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.107277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] PATH_INFO: `/v3/projects/a6efa9f078d146408eb2704ba1e05ef9/users/bc622c927b3d49a3b8c6d4836824eea5/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.125686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a6efa9f078d146408eb2704ba1e05ef9, user_id=bc622c927b3d49a3b8c6d4836824eea5, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.128644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.187745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:52.256874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 969/1954] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:52 2022] PUT /identity/v3/projects/a6efa9f078d146408eb2704ba1e05ef9/users/bc622c927b3d49a3b8c6d4836824eea5/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:52.266741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.267106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.267501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.314902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] MFA Rules not processed for user `bc622c927b3d49a3b8c6d4836824eea5`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:52.317972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:52.399640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 986/1955] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:52 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:52.401081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.495836 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2a24f4b-5dd3-42ef-8174-5a9cdb9a71e9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.504763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.505800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.506087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.506365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.507077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.509088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.578394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.579997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:52.606591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 970/1956] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:52 2022] POST /identity/v3/projects => generated 381 bytes in 206 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:52.610748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.624176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=11,keystone:SELECT=88,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:52.642649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b86c067-ee48-4d09-aeb2-532910498893 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.687692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d943ae82-e236-4dda-964c-6d4356773861 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.692792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.693475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.693760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.694039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.694361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.694772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.695018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.695307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.695591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.696824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.697364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:52.755458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.756932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.766184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 987/1957] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:52 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 188 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:52.808388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 971/1958] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:52 2022] POST /identity/v3/users => generated 297 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:52.809660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.820378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3858dd7-3461-43b1-b6d6-dc3b63fc26b4 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.825508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.826175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.826510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.826797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.827537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.829247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:52.855349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 988/1959] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:52 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:52.860674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:52.872785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53766cf-af60-44f6-8cdc-a77f146c80eb admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:52.878147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:52.879241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:52.879702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:52.880186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] PATH_INFO: `/v3/projects/c40c5db8c8a644508e988c18311d3e6d/users/555b0226a8fa4b8481a0d18e18ea10fb/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:52.896077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40c5db8c8a644508e988c18311d3e6d, user_id=555b0226a8fa4b8481a0d18e18ea10fb, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:52.899125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:53.037613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 972/1960] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:52 2022] PUT /identity/v3/projects/c40c5db8c8a644508e988c18311d3e6d/users/555b0226a8fa4b8481a0d18e18ea10fb/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:53.041558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:53.051520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6125e12e-66a9-492b-ba5b-ffc24e3cedb7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:53.133797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:53.134476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:53.134761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:53.135040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:53.135983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:53.136102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:53.138297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:53.139397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:53.149765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 989/1961] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:53 2022] GET /identity/v3/roles => generated 1420 bytes in 109 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:53.153812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:53.163441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84402481-c98c-4df0-aafd-cb9cca380b42 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:53.168579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733ee200-69c7-4c09-94f8-5795b5253bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:53.169447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ee200-69c7-4c09-94f8-5795b5253bc4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:53.176887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ee200-69c7-4c09-94f8-5795b5253bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:53.176887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733ee200-69c7-4c09-94f8-5795b5253bc4 admin admin] PATH_INFO: `/v3/projects/c40c5db8c8a644508e988c18311d3e6d/users/555b0226a8fa4b8481a0d18e18ea10fb/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:53.176887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ee200-69c7-4c09-94f8-5795b5253bc4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c40c5db8c8a644508e988c18311d3e6d, user_id=555b0226a8fa4b8481a0d18e18ea10fb, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:53.176887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733ee200-69c7-4c09-94f8-5795b5253bc4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:53.186117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 990/1962] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:53 2022] PUT /identity/v3/projects/c40c5db8c8a644508e988c18311d3e6d/users/555b0226a8fa4b8481a0d18e18ea10fb/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:53.199351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:53.205518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:53.205518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:53.205704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1696264d-ef7c-4202-9f7e-77d230f1257c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:53.228564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] MFA Rules not processed for user `555b0226a8fa4b8481a0d18e18ea10fb`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:53.229838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:53.256472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:53.257057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:53.257339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:53.257617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:53.259483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:53.299148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 991/1963] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:53 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:56:53.322285 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:53.323719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:53.334388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 973/1964] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:53 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 199 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:56:53.397341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:53.482830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:53.495493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b142d6b-2d9e-47ce-9d6d-446a63ea2409 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:53.500029 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cbd9ef0-7cde-44b9-9e5f-ac4a77d20331 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:53.500687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd9ef0-7cde-44b9-9e5f-ac4a77d20331 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:53.500974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd9ef0-7cde-44b9-9e5f-ac4a77d20331 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:53.502929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cbd9ef0-7cde-44b9-9e5f-ac4a77d20331 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:53.518122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd9ef0-7cde-44b9-9e5f-ac4a77d20331 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:53.519489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cbd9ef0-7cde-44b9-9e5f-ac4a77d20331 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:53.530018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 992/1965] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:53 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:54.143421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:54.431937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=2,keystone:DELETE=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:54.580429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:54.663495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ec77b-59ee-42e8-85ad-cc6f430aa93f service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:54.669684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:54.670583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:54.670874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:54.671153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] PATH_INFO: `/v3/users/bc622c927b3d49a3b8c6d4836824eea5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:54.677809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc622c927b3d49a3b8c6d4836824eea5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:54.677809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:54.768986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=5,keystone:SELECT=85,keystone:DELETE=10 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:54.856520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc622c927b3d49a3b8c6d4836824eea5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:54.857184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc622c927b3d49a3b8c6d4836824eea5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:54.953965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc622c927b3d49a3b8c6d4836824eea5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:54.961674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc622c927b3d49a3b8c6d4836824eea5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:54.974034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc622c927b3d49a3b8c6d4836824eea5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:54.979955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 974/1966] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:54 2022] DELETE /identity/v3/users/bc622c927b3d49a3b8c6d4836824eea5 => generated 0 bytes in 400 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:54.984929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:54.985191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:54.985425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:55.003558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:55.004800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:55.069055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 993/1967] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:55.078510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:55.080160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:55.123771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:55.136907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:55.138275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01f9538e-e4b6-4b07-a480-001530f4583b admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:55.138390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:55.142028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:55.142751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:55.143040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:55.143322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:55.152661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:55.154200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:55.164273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 975/1968] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:55.203647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0a6b9fb-99c3-4c9f-93e7-47065066a39e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:55.207677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:55.208373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:55.208820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:55.209087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:55.210920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:55.218456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:55.275989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:55.298650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:55.301918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:55.310543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 994/1969] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:55 2022] GET /identity/v3/auth/tokens => generated 3311 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:55.547572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:55.547572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:55.561643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972a3e61-514c-4af4-8a92-5bd9c5c1efd4 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:55.567742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:55.568426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:55.569726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:55.570022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:55.570353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] PATH_INFO: `/v3/projects/a6efa9f078d146408eb2704ba1e05ef9` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:55.571540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] RBAC: Authorizing `identity:delete_project(project_id=a6efa9f078d146408eb2704ba1e05ef9)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:55.578866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:55.601225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=4,keystone:DELETE=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:55.680413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a6efa9f078d146408eb2704ba1e05ef9', 'request_id': 'req-7348c5db-8680-4aab-9185-a02a1e3a492c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:55.685827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a6efa9f078d146408eb2704ba1e05ef9', 'request_id': 'req-7348c5db-8680-4aab-9185-a02a1e3a492c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:55.737010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:55.770977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a6efa9f078d146408eb2704ba1e05ef9', 'request_id': 'req-7348c5db-8680-4aab-9185-a02a1e3a492c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:55.783048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] The token cache is being invalidate because project a6efa9f078d146408eb2704ba1e05ef9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:55.783338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:55.784012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:55.785572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 976/1970] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:55 2022] DELETE /identity/v3/projects/a6efa9f078d146408eb2704ba1e05ef9 => generated 0 bytes in 243 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:55.790324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:55.800143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7de6dba6-2f45-4710-b14a-63f4da60a483 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:55.872794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:55.873859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:55.873974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:55.874247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] PATH_INFO: `/v3/users/555b0226a8fa4b8481a0d18e18ea10fb` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:55.876444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] RBAC: Authorizing `identity:delete_user(user_id=555b0226a8fa4b8481a0d18e18ea10fb)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:55.877802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:55.911960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.014492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.085324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '555b0226a8fa4b8481a0d18e18ea10fb'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.085986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '555b0226a8fa4b8481a0d18e18ea10fb'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.167927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '555b0226a8fa4b8481a0d18e18ea10fb'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.175311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '555b0226a8fa4b8481a0d18e18ea10fb'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.188082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-583c0e67-c33e-4dba-aef5-33d8a5454239 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '555b0226a8fa4b8481a0d18e18ea10fb'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.193930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 995/1971] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:56:55 2022] DELETE /identity/v3/users/555b0226a8fa4b8481a0d18e18ea10fb => generated 0 bytes in 405 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:56.334694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.400236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.493693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.583004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.592966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:56.592966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:56.604822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7348c5db-8680-4aab-9185-a02a1e3a492c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:56.664883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:56.665701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:56.666200 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:56.666667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] PATH_INFO: `/v3/projects/c40c5db8c8a644508e988c18311d3e6d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:56.667980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] RBAC: Authorizing `identity:delete_project(project_id=c40c5db8c8a644508e988c18311d3e6d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:56.670564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:56.799216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c40c5db8c8a644508e988c18311d3e6d', 'request_id': 'req-cf92a259-e5e5-43ad-b7ec-29357ce606ff'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.804445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c40c5db8c8a644508e988c18311d3e6d', 'request_id': 'req-cf92a259-e5e5-43ad-b7ec-29357ce606ff'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.845347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.873485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c40c5db8c8a644508e988c18311d3e6d', 'request_id': 'req-cf92a259-e5e5-43ad-b7ec-29357ce606ff'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.885347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] The token cache is being invalidate because project c40c5db8c8a644508e988c18311d3e6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:56:56.885347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.885988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cf92a259-e5e5-43ad-b7ec-29357ce606ff admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:56:56.887542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 977/1972] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:56:56 2022] DELETE /identity/v3/projects/c40c5db8c8a644508e988c18311d3e6d => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:56:56.894210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:56.894552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:56.894864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:56.913166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:56.914377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:56.943786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:56.977267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 996/1973] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:56.985297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:56.985864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:56.986214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.007180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:57.008686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:57.020639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 978/1974] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:57.024939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.036258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39db0734-e2f6-4d0d-8def-691a286d3a2f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.041614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.042327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.042616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.042897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.044279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:56:57.044589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.046194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.059444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 997/1975] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:56:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:56:57.067485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.077214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca41e386-4ee3-4abb-a4b7-5df9a692f7e2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.083287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.084116 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.084466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.084752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.085680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.087604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.167082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 979/1976] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:56:57 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:57.171650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.182546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babc62a9-104e-420a-a8b8-c08c6c5b97ac admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.246987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.247745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.248076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.252377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.252377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.252377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.351438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 998/1977] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:56:57 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:56:57.355386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.364753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:57.371001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630df472-b7d0-47d5-b13c-996480b06894 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.376993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.378017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.378387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.378756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.379633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.381410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.393251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 980/1978] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:57.397242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.408342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3b390db-bbfb-42a8-a207-51d27b1a3a2e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.413013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d13a9bc-1205-4e67-8843-1d36d66c1c85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.413681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13a9bc-1205-4e67-8843-1d36d66c1c85 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.416423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13a9bc-1205-4e67-8843-1d36d66c1c85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.416423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d13a9bc-1205-4e67-8843-1d36d66c1c85 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.416423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13a9bc-1205-4e67-8843-1d36d66c1c85 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.416704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d13a9bc-1205-4e67-8843-1d36d66c1c85 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.428073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 999/1979] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:56:57 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:56:57.432345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.439981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=5,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:57.442644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b93e11aa-6a08-48a5-bccc-56829122be01 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.447230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.448143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.448425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.448696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] PATH_INFO: `/v3/projects/345dc1b6fec149e88f1be7837d126d20/users/e6730f4274e240dd9da30e5c3bc3679b/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.464100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] RBAC: Authorizing `identity:create_grant(project_id=345dc1b6fec149e88f1be7837d126d20, user_id=e6730f4274e240dd9da30e5c3bc3679b, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.467459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.533228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 981/1980] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:56:57 2022] PUT /identity/v3/projects/345dc1b6fec149e88f1be7837d126d20/users/e6730f4274e240dd9da30e5c3bc3679b/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:56:57.538390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.538693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.538982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.557646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:56:57.558861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:57.626586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1000/1981] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:56:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:56:57.639009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:56:57.642045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:56:57.689571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:57.723111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba58ce51-1ab2-477d-8705-fe792816617f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:56:57.728186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52f72c69-7b6e-4ec7-b0fa-8726160d763a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:56:57.728860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f72c69-7b6e-4ec7-b0fa-8726160d763a service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:56:57.729132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f72c69-7b6e-4ec7-b0fa-8726160d763a service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:56:57.729408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52f72c69-7b6e-4ec7-b0fa-8726160d763a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:56:57.745143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f72c69-7b6e-4ec7-b0fa-8726160d763a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:56:57.747370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52f72c69-7b6e-4ec7-b0fa-8726160d763a service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:56:57.757731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 982/1982] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:56:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:56:58.401670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:58.520312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:58.701162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:59.397883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:56:59.466314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:00.534686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:00.899733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:02.397375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:02.399235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:02.428216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbad2178-42d6-40cf-bbb0-1e00f4925ac0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:02.443276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:02.444330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:02.444586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:02.444903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] PATH_INFO: `/v3/users/12fc5277226e4c50bc328aa35a2a12ff` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:02.446203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] RBAC: Authorizing `identity:delete_user(user_id=12fc5277226e4c50bc328aa35a2a12ff)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:02.448605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:02.529604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:02.654124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '12fc5277226e4c50bc328aa35a2a12ff'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:02.654906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '12fc5277226e4c50bc328aa35a2a12ff'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:02.732212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '12fc5277226e4c50bc328aa35a2a12ff'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:02.755275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '12fc5277226e4c50bc328aa35a2a12ff'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:02.796918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '12fc5277226e4c50bc328aa35a2a12ff'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:02.807080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1001/1983] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:02 2022] DELETE /identity/v3/users/12fc5277226e4c50bc328aa35a2a12ff => generated 0 bytes in 410 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:02.812060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:02.812475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:02.812826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:02.833827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:02.835117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:02.907228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 983/1984] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:02.916974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:02.919665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:03.030896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6984aa13-26fe-4e12-a90e-e01ebe403808 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:03.049091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:03.049091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:03.049091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:03.049091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:03.052461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:03.053979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:03.067916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1002/1985] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:02 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:03.724294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:03.726642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:03.747253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5339dc07-1e44-4a1a-a9ec-a687039c088c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:03.752876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:03.759696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:03.759696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:03.759696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] PATH_INFO: `/v3/projects/80a65c7474fc4e1f830606fddd3496df` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:03.759696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80a65c7474fc4e1f830606fddd3496df)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:03.759696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:03.938580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80a65c7474fc4e1f830606fddd3496df', 'request_id': 'req-7275818d-ae8c-47d5-bb4b-324cea2ef781'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:03.945994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80a65c7474fc4e1f830606fddd3496df', 'request_id': 'req-7275818d-ae8c-47d5-bb4b-324cea2ef781'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.008350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80a65c7474fc4e1f830606fddd3496df', 'request_id': 'req-7275818d-ae8c-47d5-bb4b-324cea2ef781'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.023678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] The token cache is being invalidate because project 80a65c7474fc4e1f830606fddd3496df was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:04.024062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.024727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7275818d-ae8c-47d5-bb4b-324cea2ef781 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.026293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 984/1986] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:03 2022] DELETE /identity/v3/projects/80a65c7474fc4e1f830606fddd3496df => generated 0 bytes in 303 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:04.031290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:04.042873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32d0fe4a-2757-46fd-99f1-b28237a68cd9 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:04.101742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:04.109448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:04.110490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:04.110973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:04.111262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] PATH_INFO: `/v3/users/399026bfda314e10a603382d0a533908` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:04.112577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=399026bfda314e10a603382d0a533908)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:04.114506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:04.299329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '399026bfda314e10a603382d0a533908'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.300592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '399026bfda314e10a603382d0a533908'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.374985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '399026bfda314e10a603382d0a533908'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.386072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '399026bfda314e10a603382d0a533908'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.417139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '399026bfda314e10a603382d0a533908'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:04.426518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1003/1987] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:04 2022] DELETE /identity/v3/users/399026bfda314e10a603382d0a533908 => generated 0 bytes in 396 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:04.783367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:04.783934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:04.784104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:04.814242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] MFA Rules not processed for user `e6730f4274e240dd9da30e5c3bc3679b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:04.815857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:04.911127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 985/1988] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:04.915483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:04.916919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:04.981461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96512dd1-5ad8-46f9-870b-7cc0dbf012c9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:04.988312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:04.988312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:04.988312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:04.988312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:04.988680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:04.989710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:04.990415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:04.991327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:05.012059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a66e5706-83a0-4019-a05a-2fa7687a267c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.018242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.019244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.019581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.019886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] PATH_INFO: `/v3/projects/d34b9a1c989948b08d3bf5a80c224892` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.021186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] RBAC: Authorizing `identity:delete_project(project_id=d34b9a1c989948b08d3bf5a80c224892)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.023179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.058029 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1004/1989] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:04 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:05.061998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.125098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-586a28bb-64de-49e4-9ca3-9f6cb30e6c81 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.127067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:05.131420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd34b9a1c989948b08d3bf5a80c224892', 'request_id': 'req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:05.138144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd34b9a1c989948b08d3bf5a80c224892', 'request_id': 'req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:05.184706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.185399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.185690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.185949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.186639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.188629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.268346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd34b9a1c989948b08d3bf5a80c224892', 'request_id': 'req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:05.280999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] The token cache is being invalidate because project d34b9a1c989948b08d3bf5a80c224892 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:05.281428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:05.282177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8ab071ed-4c8e-4fc1-8331-19cb64c439fa admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:05.283974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 986/1990] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:04 2022] DELETE /identity/v3/projects/d34b9a1c989948b08d3bf5a80c224892 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:05.288694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ef707-d074-4543-8f0c-4c83686df8d4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.288984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ef707-d074-4543-8f0c-4c83686df8d4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.289380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03ef707-d074-4543-8f0c-4c83686df8d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.305759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1005/1991] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:05 2022] POST /identity/v3/users => generated 298 bytes in 241 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:57:05.306920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.309219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:05.310655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c03ef707-d074-4543-8f0c-4c83686df8d4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:05.311838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c03ef707-d074-4543-8f0c-4c83686df8d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:05.319896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 987/1992] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:05.321233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9a12c6-0e30-480a-b07d-2792014f829f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.326656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.329455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.329455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.329455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.329711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.329711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.329891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.329977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.330347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.342122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1006/1993] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:57:05.346374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.351375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:05.352643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:05.359034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31b5f5b8-84e9-4774-9434-8d68c1aedf6a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.361966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 988/1994] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:05.364263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.365484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.365838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.366183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] PATH_INFO: `/v3/projects/bd0df547d2734080a1fecaf71ae4b520/users/21d97de7b77d4cd096f00c175c4a3556/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.366530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.376075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cba54ef6-20e9-469b-9026-091129d217e2 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.380858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.381521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.381809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.382087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.382310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0df547d2734080a1fecaf71ae4b520, user_id=21d97de7b77d4cd096f00c175c4a3556, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.382874 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:05.383183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.384829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.385036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.396712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 989/1995] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:57:05.401048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.411241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c968108f-be55-4af3-8325-4ea1193efb2f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.415928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.416592 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.416881 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.417160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.417919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.420402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.467819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1007/1996] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:05 2022] PUT /identity/v3/projects/bd0df547d2734080a1fecaf71ae4b520/users/21d97de7b77d4cd096f00c175c4a3556/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:05.468472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.530033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42eccce-43d2-4a0d-aae0-5a5bdbb80a52 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.533902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 990/1997] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:05 2022] POST /identity/v3/projects => generated 379 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:57:05.538308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.539190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.539915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.540206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.540491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.541292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.543118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.558678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1008/1998] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:57:05.559936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.573451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e1c04ec-2d7b-4481-a9da-496cd3ce036e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.607312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdb90d09-3d8a-4f16-89dd-b02fb77a1194 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.613246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.613929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.614243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.614604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.615342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.617454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.635128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.636067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.637279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.637279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] PATH_INFO: `/v3/projects/bd0df547d2734080a1fecaf71ae4b520/users/21d97de7b77d4cd096f00c175c4a3556/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.639231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=bd0df547d2734080a1fecaf71ae4b520, user_id=21d97de7b77d4cd096f00c175c4a3556, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.642013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.653032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1009/1999] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:05 2022] PUT /identity/v3/projects/bd0df547d2734080a1fecaf71ae4b520/users/21d97de7b77d4cd096f00c175c4a3556/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 94 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:05.694524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 991/2000] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:05 2022] POST /identity/v3/users => generated 290 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:05.695903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.782978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b611285d-6830-422d-8854-10261e1737a0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.803994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.804983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.806963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.807276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.810565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.812377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.826798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1010/2001] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 131 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:05.830426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.837591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:05.845680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5b1dd74-d3a9-4d7d-8519-1fb912917dd3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.853765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99577eda-1435-40fa-bc03-55d70dbc7612 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.854631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99577eda-1435-40fa-bc03-55d70dbc7612 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.855047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99577eda-1435-40fa-bc03-55d70dbc7612 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.855913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99577eda-1435-40fa-bc03-55d70dbc7612 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.857590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99577eda-1435-40fa-bc03-55d70dbc7612 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.862987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99577eda-1435-40fa-bc03-55d70dbc7612 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:05.877097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 992/2002] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:05 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:05.881432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:05.895099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98a9d390-8cfd-407f-9edc-90913856f889 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:05.927752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:05.928756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:05.929044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:05.930166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] PATH_INFO: `/v3/projects/b2fd01e65bf9499580abfcb2b71032c9/users/484ddd09f5f7418283a9b9a07e64010e/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:05.947143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] RBAC: Authorizing `identity:create_grant(project_id=b2fd01e65bf9499580abfcb2b71032c9, user_id=484ddd09f5f7418283a9b9a07e64010e, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:05.949945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:06.014209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1011/2003] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:05 2022] PUT /identity/v3/projects/b2fd01e65bf9499580abfcb2b71032c9/users/484ddd09f5f7418283a9b9a07e64010e/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 134 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:06.025151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.025453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.025744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.059768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] MFA Rules not processed for user `484ddd09f5f7418283a9b9a07e64010e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:06.063159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:06.131747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 993/2004] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:06.136280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:06.217166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6d774b-dd00-40ad-8f01-fb667e0d40ba admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:06.222647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:06.223713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.223921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.224223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.225004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:06.227080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:06.316143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1012/2005] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:06 2022] POST /identity/v3/projects => generated 379 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:06.317043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:06.330265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8f4b30d-0a8f-45cc-a5f5-96fe65ce4211 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:06.412326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:06.413137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.413485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.413764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.414569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:06.416885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:06.567664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 994/2006] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:06 2022] POST /identity/v3/users => generated 290 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:57:06.572016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:06.593838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9193fb-e012-457f-b790-4f01e087db35 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:06.602455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:06.603556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.604202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.605546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.606842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:06.612399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:06.626005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1013/2007] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:57:06.629650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:06.640429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7555a9aa-960c-4d81-b043-7acab043d7c5 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:06.643352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:06.648303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c13354-a4ba-4060-adb2-541fc8f2a6f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:06.649620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c13354-a4ba-4060-adb2-541fc8f2a6f3 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.653856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c13354-a4ba-4060-adb2-541fc8f2a6f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.653856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c13354-a4ba-4060-adb2-541fc8f2a6f3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.653856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c13354-a4ba-4060-adb2-541fc8f2a6f3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:06.653856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c13354-a4ba-4060-adb2-541fc8f2a6f3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:06.668028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 995/2008] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:06 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:06.668675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:06.692590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6661d836-c2a0-4442-a41a-69579b216718 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:06.699275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:06.700453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.702339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.702339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] PATH_INFO: `/v3/projects/d6cc5f94687446f58bc25523ece4fb69/users/e63c6825c1074c7a87cbce7d6e4e85cf/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.732283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d6cc5f94687446f58bc25523ece4fb69, user_id=e63c6825c1074c7a87cbce7d6e4e85cf, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:06.735536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:06.879459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1014/2009] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:06 2022] PUT /identity/v3/projects/d6cc5f94687446f58bc25523ece4fb69/users/e63c6825c1074c7a87cbce7d6e4e85cf/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:06.890965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:06.891479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:06.891590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:06.920563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] MFA Rules not processed for user `e63c6825c1074c7a87cbce7d6e4e85cf`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:06.921867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:07.050212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 996/2010] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:06 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:07.055199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:07.219250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73c89f8-d06e-49d1-b0c0-07b64d3ff772 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:07.237379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:07.238296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:07.238454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:07.238734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:07.239440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:07.241377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:07.372253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1015/2011] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:07 2022] POST /identity/v3/projects => generated 375 bytes in 317 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:07.376231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:07.412636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928e82ee-d0ba-4053-be44-f2fa89b7c997 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:07.498164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:07.620619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:07.621347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:07.621642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:07.621928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:07.625901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:07.628985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:07.831977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 997/2012] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:07 2022] POST /identity/v3/users => generated 294 bytes in 456 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:07.836338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:07.868623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95f5f25d-e7b9-4bfc-ad61-a3897c500d31 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:07.877834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:07.879023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:07.879535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:07.880098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:07.881450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:07.883429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:07.900489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1016/2013] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:07 2022] GET /identity/v3/roles => generated 1420 bytes in 65 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:07.905866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:07.928858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-576b316a-f3f8-4e0c-b384-dec7c2c1a7d7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:07.956411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:07.957630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:07.958083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:07.963827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] PATH_INFO: `/v3/projects/7f5f093b00f44c27803d6f79d64f35ab/users/0f28679728a7493fba2a5cf1c8cbaa82/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:07.998294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5f093b00f44c27803d6f79d64f35ab, user_id=0f28679728a7493fba2a5cf1c8cbaa82, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:08.001315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:08.019910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:08.023193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:08.048945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5abdf296-7770-485c-b624-6224e1642622 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:08.072573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:08.073678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.074632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.075052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] PATH_INFO: `/v3/users/95496ebf94b046ff8d64c27ed2243e28` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.076641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=95496ebf94b046ff8d64c27ed2243e28)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:08.080407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:08.080657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 998/2014] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:07 2022] PUT /identity/v3/projects/7f5f093b00f44c27803d6f79d64f35ab/users/0f28679728a7493fba2a5cf1c8cbaa82/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:08.084180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:08.171828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36628acd-8761-43a7-84c0-42af2519fd93 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:08.177832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:08.178811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.179291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.179713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.182647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:08.183222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:08.195942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 999/2015] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:08 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:08.200286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:08.221530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eda08249-5f8d-44d5-87a8-1ca84e2f79d9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:08.232938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1622c845-dcbc-4787-97c1-bab240c77d4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:08.234506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c845-dcbc-4787-97c1-bab240c77d4c admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.234797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c845-dcbc-4787-97c1-bab240c77d4c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.235076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1622c845-dcbc-4787-97c1-bab240c77d4c admin admin] PATH_INFO: `/v3/projects/7f5f093b00f44c27803d6f79d64f35ab/users/0f28679728a7493fba2a5cf1c8cbaa82/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.237406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622c845-dcbc-4787-97c1-bab240c77d4c admin admin] RBAC: Authorizing `identity:create_grant(project_id=7f5f093b00f44c27803d6f79d64f35ab, user_id=0f28679728a7493fba2a5cf1c8cbaa82, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:08.240126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1622c845-dcbc-4787-97c1-bab240c77d4c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:08.272310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1000/2016] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:08 2022] PUT /identity/v3/projects/7f5f093b00f44c27803d6f79d64f35ab/users/0f28679728a7493fba2a5cf1c8cbaa82/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 73 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:08.286206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.286498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.289103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.331134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '95496ebf94b046ff8d64c27ed2243e28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:08.332612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '95496ebf94b046ff8d64c27ed2243e28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:08.372234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] MFA Rules not processed for user `0f28679728a7493fba2a5cf1c8cbaa82`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:08.373864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:08.378700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=3,keystone:SELECT=92,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:08.437358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '95496ebf94b046ff8d64c27ed2243e28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:08.452669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '95496ebf94b046ff8d64c27ed2243e28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:08.501644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f9c0634-5a3a-495a-b333-b05cae3efe43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '95496ebf94b046ff8d64c27ed2243e28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:08.522636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1017/2017] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:08 2022] DELETE /identity/v3/users/95496ebf94b046ff8d64c27ed2243e28 => generated 0 bytes in 501 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:08.525476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.525768 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.526051 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.536091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1001/2018] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3309 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:08.595090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:08.603449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:08.611288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:08.612845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:08.713745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1018/2019] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:08.726657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:08.728670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:08.765296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa147c5a-b5bd-4b1f-bf7a-7c5866b166ce None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:08.847190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:08.848835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.849400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.849827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.851937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:08.872104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99f23399-2f31-452f-857e-c7cbaf32b712 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:08.882950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:08.883709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:08.884098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:08.884391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:08.905450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:08.907857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:08.934130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1019/2020] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:08.943426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:08.968985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:08.971543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:08.992125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1002/2021] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:08 2022] GET /identity/v3/auth/tokens => generated 3309 bytes in 382 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:09.151688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:09.154256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:09.171174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7e8d7e5-15a2-4b49-bdc2-dfea578433da service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:09.175431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:09.176189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:09.176479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:09.176795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:09.178844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:09.327912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:09.329687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:09.351149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1020/2022] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:09 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:09.882799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:09.882799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:10.004644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6466472-b397-41a9-81bc-2854342a424a service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:10.004644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:10.004644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:10.004644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:10.006467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] PATH_INFO: `/v3/projects/a23c543d8b4341e6a0d9d3e7bad8acd1` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:10.006467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a23c543d8b4341e6a0d9d3e7bad8acd1)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:10.006467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:10.182854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a23c543d8b4341e6a0d9d3e7bad8acd1', 'request_id': 'req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.190723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a23c543d8b4341e6a0d9d3e7bad8acd1', 'request_id': 'req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.450172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a23c543d8b4341e6a0d9d3e7bad8acd1', 'request_id': 'req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.467103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] The token cache is being invalidate because project a23c543d8b4341e6a0d9d3e7bad8acd1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:10.469322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.472639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.474614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1003/2023] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:09 2022] DELETE /identity/v3/projects/a23c543d8b4341e6a0d9d3e7bad8acd1 => generated 0 bytes in 602 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:10.478359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:10.490978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-039b3745-0758-468f-8369-8a62fa0c78fe service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:10.551180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:10.552379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:10.552732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:10.553009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] PATH_INFO: `/v3/users/bb063aa068844f12899649f23ab79962` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:10.554407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bb063aa068844f12899649f23ab79962)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:10.556448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:10.807173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bb063aa068844f12899649f23ab79962'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.808048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bb063aa068844f12899649f23ab79962'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.888681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bb063aa068844f12899649f23ab79962'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.896572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bb063aa068844f12899649f23ab79962'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.911424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bb063aa068844f12899649f23ab79962'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:10.923500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1021/2024] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:10 2022] DELETE /identity/v3/users/bb063aa068844f12899649f23ab79962 => generated 0 bytes in 442 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:11.142554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:11.260951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:11.404053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:11.406008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:11.430105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33e0f7ef-82c2-4e42-9d98-7fcad81e9263 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:11.527722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:11.531292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:11.532180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:11.533695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] PATH_INFO: `/v3/projects/64040e0cca9346cbb2d47ef165ef62c3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:11.535541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] RBAC: Authorizing `identity:delete_project(project_id=64040e0cca9346cbb2d47ef165ef62c3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:11.537886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:11.655354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '64040e0cca9346cbb2d47ef165ef62c3', 'request_id': 'req-6e2aa119-e6ec-4271-9bb1-88c82ca31363'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:11.665305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '64040e0cca9346cbb2d47ef165ef62c3', 'request_id': 'req-6e2aa119-e6ec-4271-9bb1-88c82ca31363'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:11.757030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '64040e0cca9346cbb2d47ef165ef62c3', 'request_id': 'req-6e2aa119-e6ec-4271-9bb1-88c82ca31363'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:11.783130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] The token cache is being invalidate because project 64040e0cca9346cbb2d47ef165ef62c3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:11.785010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:11.786522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:11.788672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1004/2025] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:11 2022] DELETE /identity/v3/projects/64040e0cca9346cbb2d47ef165ef62c3 => generated 0 bytes in 385 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:11.792651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:11.794179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:11.807998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53520ac2-0fcb-466b-834f-16b5b521ece4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:11.928221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:11.929558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:11.929989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:11.930461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] PATH_INFO: `/v3/users/24caffffcb1e4d2b800e06b7a7a5d55a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:11.931774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=24caffffcb1e4d2b800e06b7a7a5d55a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:11.937881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:12.130675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '24caffffcb1e4d2b800e06b7a7a5d55a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:12.131610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '24caffffcb1e4d2b800e06b7a7a5d55a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:12.225903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '24caffffcb1e4d2b800e06b7a7a5d55a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:12.238217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '24caffffcb1e4d2b800e06b7a7a5d55a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:12.252656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-baa5434e-2e26-4416-8adb-61f56f94cb3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '24caffffcb1e4d2b800e06b7a7a5d55a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:12.265062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1022/2026] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:11 2022] DELETE /identity/v3/users/24caffffcb1e4d2b800e06b7a7a5d55a => generated 0 bytes in 468 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:12.488718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:12.790132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:12.930290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:12.932364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:12.947893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e2aa119-e6ec-4271-9bb1-88c82ca31363 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:13.034025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:13.035550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.035959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.036303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] PATH_INFO: `/v3/projects/3cc2f370bae44d43a1996900eb77a703` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.037685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] RBAC: Authorizing `identity:delete_project(project_id=3cc2f370bae44d43a1996900eb77a703)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:13.040125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:13.177839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3cc2f370bae44d43a1996900eb77a703', 'request_id': 'req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:13.198637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3cc2f370bae44d43a1996900eb77a703', 'request_id': 'req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:13.288497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3cc2f370bae44d43a1996900eb77a703', 'request_id': 'req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:13.311726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] The token cache is being invalidate because project 3cc2f370bae44d43a1996900eb77a703 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:13.312338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:13.313369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8bf1f3f6-39bf-4956-8d9c-7b3c27ec016b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:13.322422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1005/2027] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:12 2022] DELETE /identity/v3/projects/3cc2f370bae44d43a1996900eb77a703 => generated 0 bytes in 387 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:13.325720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.326205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.326284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.358604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:13.359900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:13.467172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1023/2028] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:13.475937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.476467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.476601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.502739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:13.504168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:13.513538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1006/2029] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:13.517981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:13.529363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daca03ca-c5c3-4ef8-b16a-8d7819d5eb42 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:13.538740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:13.539822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.540413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.540914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.542357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:13.543434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:13.548737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:13.577222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:13.578559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1024/2030] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:57:13.582903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:13.594664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea6ae2-1f58-4df6-b65c-e002de67f34d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:13.601307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dc69f6-18bc-4150-96d7-6057ac484264 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:13.601993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc69f6-18bc-4150-96d7-6057ac484264 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.602284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc69f6-18bc-4150-96d7-6057ac484264 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.602566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dc69f6-18bc-4150-96d7-6057ac484264 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.603273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc69f6-18bc-4150-96d7-6057ac484264 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:13.607184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dc69f6-18bc-4150-96d7-6057ac484264 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:13.745549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1007/2031] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:13 2022] POST /identity/v3/projects => generated 371 bytes in 163 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:13.749846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:13.762449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db60b84-6beb-4cbf-881f-3b85d64553e3 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:13.775220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.775508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.775794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.803906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=14,keystone:SELECT=84,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:13.818813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] MFA Rules not processed for user `21d97de7b77d4cd096f00c175c4a3556`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:13.822161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:13.848457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:13.849175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:13.849466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:13.849745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:13.850889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:13.853034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:13.926887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1008/2032] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:13 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:13.942486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:13.944063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:13.988370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1025/2033] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:13 2022] POST /identity/v3/users => generated 286 bytes in 239 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:13.997254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:14.016575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6252f0-0e13-4fae-828f-11d1c5b134f0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:14.022146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:14.023300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.023732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.024270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.025156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:14.026950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:14.033736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e0ab572-c35f-402b-9e2a-5cfbad43aa82 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:14.040455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:14.041644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.042102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.043315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1026/2034] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:13 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:14.044191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.049699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:14.049820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:14.059650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31d5bae7-d58c-4854-ba15-330faebdb16e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:14.066019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:14.066877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.067336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.067773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.068700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:14.070868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:14.083382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1027/2035] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:14 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:14.087174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:14.108020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6736ea7a-16d3-4e42-809a-191aa3413893 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:14.113771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:14.115037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.123901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.127937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] PATH_INFO: `/v3/projects/f431b3b8e5534e31ab70a9ab791e36e1/users/d18e9774031e49a2aaf213bb070758bc/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.142426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:14.144142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b57ec0c-9d9f-4546-9cfd-d6ad920a6b83 service glance] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:14.147704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f431b3b8e5534e31ab70a9ab791e36e1, user_id=d18e9774031e49a2aaf213bb070758bc, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:14.151719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:14.154519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1009/2036] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:13 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 213 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:14.274655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1028/2037] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:14 2022] PUT /identity/v3/projects/f431b3b8e5534e31ab70a9ab791e36e1/users/d18e9774031e49a2aaf213bb070758bc/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:14.284703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8156589-c172-4ed2-bf81-6bbc444ca28e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.285713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8156589-c172-4ed2-bf81-6bbc444ca28e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.286211 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8156589-c172-4ed2-bf81-6bbc444ca28e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.322473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d8156589-c172-4ed2-bf81-6bbc444ca28e None None] MFA Rules not processed for user `d18e9774031e49a2aaf213bb070758bc`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:14.324750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d8156589-c172-4ed2-bf81-6bbc444ca28e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:14.404524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1010/2038] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:14 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:14.418629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:14.481492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.481840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.482129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.485128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098fe9f6-6a08-4ca0-b50e-f1c3e236ffa4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:14.489161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:14.489821 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.490107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.490386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.502341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:14.503750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:14.508219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:14.509552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:14.514347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1029/2039] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:14 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:14.580364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1011/2040] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:57:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:14.600742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:14.600742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:14.692418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61202b8-73e1-4297-b0e6-3db9702336f1 service glance] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:14.697821 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4186ff-41da-4322-89ff-fbe7ef771c86 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:14.698509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4186ff-41da-4322-89ff-fbe7ef771c86 service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:14.699319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4186ff-41da-4322-89ff-fbe7ef771c86 service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:14.699804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4186ff-41da-4322-89ff-fbe7ef771c86 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:14.719438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4186ff-41da-4322-89ff-fbe7ef771c86 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:14.720565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4186ff-41da-4322-89ff-fbe7ef771c86 service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:14.732230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1030/2041] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:57:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:57:14.834711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:15.369513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:16.363399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=1,keystone:SELECT=98,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:17.046295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:17.048130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:17.183617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f1833a-f85f-49dc-bad7-a65191a93362 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:17.190532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:17.191809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:17.197178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:17.197178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] PATH_INFO: `/v3/users/484ddd09f5f7418283a9b9a07e64010e` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:17.197178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] RBAC: Authorizing `identity:delete_user(user_id=484ddd09f5f7418283a9b9a07e64010e)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:17.197178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:17.456244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '484ddd09f5f7418283a9b9a07e64010e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:17.460264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '484ddd09f5f7418283a9b9a07e64010e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:17.577456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '484ddd09f5f7418283a9b9a07e64010e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:17.600916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '484ddd09f5f7418283a9b9a07e64010e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:17.618393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '484ddd09f5f7418283a9b9a07e64010e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:17.630704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1012/2042] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:17 2022] DELETE /identity/v3/users/484ddd09f5f7418283a9b9a07e64010e => generated 0 bytes in 581 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:17.635002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:17.635002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:17.635002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:17.639241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:17.657122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:17.659344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:17.869223 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:17.871440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:17.874913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1031/2043] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:17.989580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2014fea2-5ad3-42e8-a462-fae0c94b6623 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:17.994996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:17.999307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:17.999307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:17.999307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:18.010662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:18.013036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:18.045362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1013/2044] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:19.065973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:19.067501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:19.087002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7c30b7e-17ba-4dae-9d69-7b6d72f89efd None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:19.093192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:19.096194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:19.096509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:19.096794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] PATH_INFO: `/v3/projects/b2fd01e65bf9499580abfcb2b71032c9` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:19.098291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b2fd01e65bf9499580abfcb2b71032c9)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:19.102009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:19.257008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b2fd01e65bf9499580abfcb2b71032c9', 'request_id': 'req-d0d6c95d-e096-4697-9604-8df861ba4dc7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.264574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b2fd01e65bf9499580abfcb2b71032c9', 'request_id': 'req-d0d6c95d-e096-4697-9604-8df861ba4dc7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.270554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b2fd01e65bf9499580abfcb2b71032c9', 'request_id': 'req-d0d6c95d-e096-4697-9604-8df861ba4dc7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.363399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] The token cache is being invalidate because project b2fd01e65bf9499580abfcb2b71032c9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:19.363695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.365528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.367576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1032/2045] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:19 2022] DELETE /identity/v3/projects/b2fd01e65bf9499580abfcb2b71032c9 => generated 0 bytes in 302 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:19.376298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:19.388466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20782eaa-63f3-4fea-8fc0-37ad684ccd48 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:19.451096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:19.452025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:19.452315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:19.452590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] PATH_INFO: `/v3/users/e63c6825c1074c7a87cbce7d6e4e85cf` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:19.453800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e63c6825c1074c7a87cbce7d6e4e85cf)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:19.455745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:19.642044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e63c6825c1074c7a87cbce7d6e4e85cf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.642909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e63c6825c1074c7a87cbce7d6e4e85cf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.728047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e63c6825c1074c7a87cbce7d6e4e85cf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.734475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=85,keystone:DELETE=13,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:19.738150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e63c6825c1074c7a87cbce7d6e4e85cf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.754577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e63c6825c1074c7a87cbce7d6e4e85cf'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:19.769907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1014/2046] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:19 2022] DELETE /identity/v3/users/e63c6825c1074c7a87cbce7d6e4e85cf => generated 0 bytes in 394 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:20.372489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:20.372489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:20.396640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d6c95d-e096-4697-9604-8df861ba4dc7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:20.498112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:20.547999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:20.549162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:20.551092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:20.551092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] PATH_INFO: `/v3/projects/d6cc5f94687446f58bc25523ece4fb69` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:20.558684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d6cc5f94687446f58bc25523ece4fb69)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:20.563016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:20.698063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd6cc5f94687446f58bc25523ece4fb69', 'request_id': 'req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:20.706413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd6cc5f94687446f58bc25523ece4fb69', 'request_id': 'req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:20.712655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd6cc5f94687446f58bc25523ece4fb69', 'request_id': 'req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:20.813663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] The token cache is being invalidate because project d6cc5f94687446f58bc25523ece4fb69 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:20.813932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:20.814560 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:20.816445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1033/2047] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:20 2022] DELETE /identity/v3/projects/d6cc5f94687446f58bc25523ece4fb69 => generated 0 bytes in 449 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:20.820705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:20.822276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:20.837610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84f45421-c801-4601-83c5-d73378137b10 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:20.923393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:20.925677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:20.926194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:20.926696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] PATH_INFO: `/v3/users/0f28679728a7493fba2a5cf1c8cbaa82` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:20.928180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] RBAC: Authorizing `identity:delete_user(user_id=0f28679728a7493fba2a5cf1c8cbaa82)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:20.930464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:21.155296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0f28679728a7493fba2a5cf1c8cbaa82'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:21.156279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0f28679728a7493fba2a5cf1c8cbaa82'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:21.227680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0f28679728a7493fba2a5cf1c8cbaa82'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:21.241034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0f28679728a7493fba2a5cf1c8cbaa82'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:21.289842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7a9ebfd2-c00b-47af-8187-243ebc07b47a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0f28679728a7493fba2a5cf1c8cbaa82'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:21.294860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1015/2048] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:20 2022] DELETE /identity/v3/users/0f28679728a7493fba2a5cf1c8cbaa82 => generated 0 bytes in 475 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:21.821274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:21.822750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:21.847675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78efcc6a-24d1-4dd0-a17e-6618a9f6b8e4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:21.928663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:21.929555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:21.929886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:21.930171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] PATH_INFO: `/v3/projects/7f5f093b00f44c27803d6f79d64f35ab` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:21.932191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f5f093b00f44c27803d6f79d64f35ab)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:21.935052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:22.060234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f5f093b00f44c27803d6f79d64f35ab', 'request_id': 'req-55b8de1f-1456-47e2-90ce-1a0d3805e886'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:22.069065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f5f093b00f44c27803d6f79d64f35ab', 'request_id': 'req-55b8de1f-1456-47e2-90ce-1a0d3805e886'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:22.074789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:22.077567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f5f093b00f44c27803d6f79d64f35ab', 'request_id': 'req-55b8de1f-1456-47e2-90ce-1a0d3805e886'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:22.171600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] The token cache is being invalidate because project 7f5f093b00f44c27803d6f79d64f35ab was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:22.173133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:22.173360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-55b8de1f-1456-47e2-90ce-1a0d3805e886 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:22.179918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1034/2049] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:21 2022] DELETE /identity/v3/projects/7f5f093b00f44c27803d6f79d64f35ab => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:22.182475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:22.182953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:22.183324 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:22.205960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:22.207362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:22.295041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:22.317989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1016/2050] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 136 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:22.340477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:22.341209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:22.341662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:22.418377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:22.418377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:22.422681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1035/2051] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:22.426205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:22.442765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed87b5fe-1893-4c7c-9a41-739b1dec617e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:22.475527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:22.476250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:22.476539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:22.476818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:22.477575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:22.477889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:22.479569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:22.544790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1017/2052] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:57:22.549404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:22.560546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4ffd61c-5930-416e-a1cc-a5e86c2f4b2e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:22.571861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:22.572830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:22.574027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:22.576602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:22.577459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:22.579685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:23.096914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1036/2053] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:22 2022] POST /identity/v3/projects => generated 375 bytes in 548 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:23.111363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:23.128587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a052ad27-4bd4-4a9f-b029-578a746718dc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:23.193508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:23.194199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:23.194791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:23.195188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:23.195921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:23.197870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:23.317262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1018/2054] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:23 2022] POST /identity/v3/users => generated 288 bytes in 207 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:23.321366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:23.332395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02effb4a-e755-4068-959f-534802b451eb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:23.340353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:23.340353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:23.340353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:23.340353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:23.340711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:23.340866 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:23.352837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1037/2055] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:23.356847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:23.368509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ec6d71b-41f0-447d-baba-f079b411e477 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:23.382691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b79a364b-0269-48d8-88f5-0481cae08360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:23.383392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a364b-0269-48d8-88f5-0481cae08360 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:23.383682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a364b-0269-48d8-88f5-0481cae08360 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:23.384053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b79a364b-0269-48d8-88f5-0481cae08360 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:23.384760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a364b-0269-48d8-88f5-0481cae08360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:23.387250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b79a364b-0269-48d8-88f5-0481cae08360 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:23.404703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1019/2056] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:23 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:23.412582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:23.424055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3914e2cc-1c2b-4149-8393-e718505f2935 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:23.429078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:23.431042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:23.431042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:23.431042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] PATH_INFO: `/v3/projects/33d15c822d3541ca98ead932b369c0eb/users/16a778d984d24024b8286acfb9ac48e3/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:23.448125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] RBAC: Authorizing `identity:create_grant(project_id=33d15c822d3541ca98ead932b369c0eb, user_id=16a778d984d24024b8286acfb9ac48e3, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:23.451470 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:23.529424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1038/2057] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:23 2022] PUT /identity/v3/projects/33d15c822d3541ca98ead932b369c0eb/users/16a778d984d24024b8286acfb9ac48e3/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:23.543749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7874097-b6af-4f78-bca5-84df105468f6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:23.544350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7874097-b6af-4f78-bca5-84df105468f6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:23.544878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7874097-b6af-4f78-bca5-84df105468f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:23.576724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b7874097-b6af-4f78-bca5-84df105468f6 None None] MFA Rules not processed for user `16a778d984d24024b8286acfb9ac48e3`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:23.578264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b7874097-b6af-4f78-bca5-84df105468f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:23.692426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1020/2058] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:23 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 150 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:25.819080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:25.819080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:25.881222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90dd5ea-300c-4665-980f-2ebb9c30a70f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:25.887741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:25.888767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:25.889119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:25.889463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] PATH_INFO: `/v3/users/82e0f549da3e4ed9a7fe4f488c652b1b` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:25.890907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] RBAC: Authorizing `identity:delete_user(user_id=82e0f549da3e4ed9a7fe4f488c652b1b)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:25.893158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:26.528858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82e0f549da3e4ed9a7fe4f488c652b1b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:26.529741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82e0f549da3e4ed9a7fe4f488c652b1b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:26.602113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82e0f549da3e4ed9a7fe4f488c652b1b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:26.610549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82e0f549da3e4ed9a7fe4f488c652b1b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:26.624016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82e0f549da3e4ed9a7fe4f488c652b1b'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:26.635313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1039/2059] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:25 2022] DELETE /identity/v3/users/82e0f549da3e4ed9a7fe4f488c652b1b => generated 0 bytes in 907 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:26.640302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:26.640864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:26.640969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:26.658932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:26.662765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:26.742504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:26.779923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1021/2060] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:26.787232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:26.788817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:26.795369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=12,keystone:SELECT=84,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:26.872089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e497d1b5-b16c-4e0e-a378-959c98952589 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:26.880228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:26.881079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:26.882455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:26.882847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:26.896454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:26.898201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:26.931421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1040/2061] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:27.464791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:27.466743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:27.484381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d5edd29-1c21-4f84-bcd5-7d80836ff82c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:27.489362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:27.490235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:27.490527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:27.490809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] PATH_INFO: `/v3/projects/6c61cc884ce540c6afc8d4723e04299f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:27.492083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=6c61cc884ce540c6afc8d4723e04299f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:27.494900 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:27.611450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6c61cc884ce540c6afc8d4723e04299f', 'request_id': 'req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:27.627206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6c61cc884ce540c6afc8d4723e04299f', 'request_id': 'req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:27.758037 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6c61cc884ce540c6afc8d4723e04299f', 'request_id': 'req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:27.778671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:27.780137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:27.781919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] The token cache is being invalidate because project 6c61cc884ce540c6afc8d4723e04299f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:27.782212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:27.783357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:27.786644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1022/2062] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:27 2022] DELETE /identity/v3/projects/6c61cc884ce540c6afc8d4723e04299f => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:27.797345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:27.801473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:27.913489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5319a3d1-402b-42ee-87e0-298d67075e58 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:27.930244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b88be05-86db-4e4e-98b0-6a5ef819b0bd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:27.939828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:27.940780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:27.941045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:27.941382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] PATH_INFO: `/v3/users/c6c188648fbd4af0b980a717c1a3d6e2` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:27.942717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c6c188648fbd4af0b980a717c1a3d6e2)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:27.944912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:27.987002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:27.987662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:27.987997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:27.988333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:27.990157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:28.050980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:28.052485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08571b3d-8524-4270-9029-f4f629b8312f service glance] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:28.064501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1041/2063] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:27 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 286 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:28.067334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:28.067813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:28.068278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:28.080839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c6c188648fbd4af0b980a717c1a3d6e2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:28.081712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c6c188648fbd4af0b980a717c1a3d6e2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:28.097120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] MFA Rules not processed for user `e6730f4274e240dd9da30e5c3bc3679b`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:28.098435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:28.117444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:28.169111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c6c188648fbd4af0b980a717c1a3d6e2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:28.191396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1042/2064] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:28 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:28.202122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c6c188648fbd4af0b980a717c1a3d6e2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:28.225972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:28.227282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bae6c02-50e3-40e1-bb63-eea2617c40c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c6c188648fbd4af0b980a717c1a3d6e2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:28.244793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1023/2065] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:27 2022] DELETE /identity/v3/users/c6c188648fbd4af0b980a717c1a3d6e2 => generated 0 bytes in 445 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:29.327758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:29.340941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c412d69c-db7c-4573-9428-59e5d8d3ddca None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:29.420047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:29.420959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:29.421376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:29.421703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] PATH_INFO: `/v3/projects/19a66ac4cbf84fd586a59bc8eaa759ca` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:29.423028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=19a66ac4cbf84fd586a59bc8eaa759ca)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:29.424985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:29.546438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '19a66ac4cbf84fd586a59bc8eaa759ca', 'request_id': 'req-17a56ea3-83ae-459b-823d-24d0f23245a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:29.553488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '19a66ac4cbf84fd586a59bc8eaa759ca', 'request_id': 'req-17a56ea3-83ae-459b-823d-24d0f23245a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:29.560786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '19a66ac4cbf84fd586a59bc8eaa759ca', 'request_id': 'req-17a56ea3-83ae-459b-823d-24d0f23245a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:29.644545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] The token cache is being invalidate because project 19a66ac4cbf84fd586a59bc8eaa759ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:29.644843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:29.645482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-17a56ea3-83ae-459b-823d-24d0f23245a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:29.647640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1043/2066] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:29 2022] DELETE /identity/v3/projects/19a66ac4cbf84fd586a59bc8eaa759ca => generated 0 bytes in 320 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:29.652413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:29.653020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:29.653381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:29.672529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:29.673854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:29.768010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1024/2067] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:29.769899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:29.770191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:29.770475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:29.789262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:29.790556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:29.799533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1044/2068] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:29.803859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:29.814445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82ebcc5f-5bed-4899-a913-4529d2bb38d7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:29.819217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:29.819981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:29.820234 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:29.820512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:29.821247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:29.821559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:29.823206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:29.846045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1025/2069] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:57:29.847779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:29.858254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a3118b-b654-414a-b81d-bc30c70eb9ad None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:29.863258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:29.864053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:29.868713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:29.868713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:29.868713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:29.868713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:30.006047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1045/2070] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:29 2022] POST /identity/v3/projects => generated 373 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:30.007375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:30.019352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d533d5e0-ea1d-4488-9be7-118080504e18 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:30.108593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:30.109129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.109478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.109914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.110866 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:30.113147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:30.231187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1026/2071] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:30 2022] POST /identity/v3/users => generated 287 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:30.232416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:30.245458 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e9e3f06-9f86-4d6c-9215-72e2973adebb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:30.255565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:30.256684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.257225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.257898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.259266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:30.262473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:30.277942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1046/2072] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:30.281695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:30.301712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7747e92-9eae-4e33-a254-d467452a45d2 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:30.325808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9052cf1-b0a5-466c-a433-45045e037d79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:30.326520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9052cf1-b0a5-466c-a433-45045e037d79 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.326926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9052cf1-b0a5-466c-a433-45045e037d79 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.327225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9052cf1-b0a5-466c-a433-45045e037d79 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.328000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9052cf1-b0a5-466c-a433-45045e037d79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:30.329635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9052cf1-b0a5-466c-a433-45045e037d79 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:30.343726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1027/2073] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:30 2022] GET /identity/v3/roles => generated 1420 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:30.349125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:30.362111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4a1f69-8d50-4608-992a-98b75a2f07ef admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:30.367154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:30.368377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.368835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.369271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] PATH_INFO: `/v3/projects/7a96686d04894ed08b57af0320f43a88/users/d1674c7ea0d74218abafe63ff9f7f1f4/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.385067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:30.396080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] RBAC: Authorizing `identity:create_grant(project_id=7a96686d04894ed08b57af0320f43a88, user_id=d1674c7ea0d74218abafe63ff9f7f1f4, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:30.400515 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:30.514825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1047/2074] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:30 2022] PUT /identity/v3/projects/7a96686d04894ed08b57af0320f43a88/users/d1674c7ea0d74218abafe63ff9f7f1f4/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:30.525078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.525369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.525652 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.562970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] MFA Rules not processed for user `d1674c7ea0d74218abafe63ff9f7f1f4`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:30.564253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:30.625906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=1,keystone:SELECT=96,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:30.647207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1028/2075] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:30.660256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:30.662163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:30.748295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-940d31f8-62ed-4403-af3a-0b2df7ebd908 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:30.752693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ea266e4-0f5b-48ec-8b1a-41a906856d14 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:30.753355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea266e4-0f5b-48ec-8b1a-41a906856d14 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.753642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea266e4-0f5b-48ec-8b1a-41a906856d14 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.756503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ea266e4-0f5b-48ec-8b1a-41a906856d14 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.765564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea266e4-0f5b-48ec-8b1a-41a906856d14 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:30.766955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ea266e4-0f5b-48ec-8b1a-41a906856d14 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:30.778352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1048/2076] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:30 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 119 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:30.798869 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:30.884010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c644a0ab-5e4f-4b9e-a24e-5a2f20355353 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:30.890771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:30.892078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:30.892364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:30.892640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] PATH_INFO: `/v3/users/d1674c7ea0d74218abafe63ff9f7f1f4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:30.893934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d1674c7ea0d74218abafe63ff9f7f1f4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:30.896151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:31.107219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd1674c7ea0d74218abafe63ff9f7f1f4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.107412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd1674c7ea0d74218abafe63ff9f7f1f4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.210103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd1674c7ea0d74218abafe63ff9f7f1f4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.218725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd1674c7ea0d74218abafe63ff9f7f1f4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.223445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a98a8-6a15-4a34-b9c9-f2859e85504d None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:31.224030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a98a8-6a15-4a34-b9c9-f2859e85504d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:31.224386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043a98a8-6a15-4a34-b9c9-f2859e85504d None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:31.225959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1049/2077] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:57:31 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:57:31.234596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f119e30d-926d-4447-b626-929249e35575 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd1674c7ea0d74218abafe63ff9f7f1f4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.242650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:31.243992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:31.245562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:31.253138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1029/2078] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:30 2022] DELETE /identity/v3/users/d1674c7ea0d74218abafe63ff9f7f1f4 => generated 0 bytes in 452 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:31.255802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:31.256280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:31.256744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:31.271431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] MFA Rules not processed for user `f482060b5ab3458ba78d94184d68f10d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:31.272719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:31.283192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:31.285250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:31.348975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1030/2079] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:31.359901 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:31.360049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1050/2080] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:57:31 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:31.361212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:31.369996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:31.371352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:31.384662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4895681a-b80a-4dcd-af26-f5052fb08668 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:31.392109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358034-982d-4360-b32c-939763738b5f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:31.392922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358034-982d-4360-b32c-939763738b5f service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:31.393213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358034-982d-4360-b32c-939763738b5f service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:31.393502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29358034-982d-4360-b32c-939763738b5f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:31.410443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29358034-982d-4360-b32c-939763738b5f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:31.410443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29358034-982d-4360-b32c-939763738b5f service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:31.422074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1051/2081] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:31 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:31.422531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358034-982d-4360-b32c-939763738b5f service placement] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:31.424131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-29358034-982d-4360-b32c-939763738b5f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:31.433406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-509c5cd5-702e-494a-b650-926b3203c0d8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:31.438117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:31.438778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:31.439065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:31.439346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:31.439558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29358034-982d-4360-b32c-939763738b5f service placement] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:31.445461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:31.446489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:31.446785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:31.447071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] PATH_INFO: `/v3/users/fae075fce4d94d679a8dc5f508a1c3cf` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:31.448562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fae075fce4d94d679a8dc5f508a1c3cf)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:31.450766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:31.451405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:31.452027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:31.462340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1031/2082] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:31.519135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:31.641681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fae075fce4d94d679a8dc5f508a1c3cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.642475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fae075fce4d94d679a8dc5f508a1c3cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.762295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fae075fce4d94d679a8dc5f508a1c3cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.770446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fae075fce4d94d679a8dc5f508a1c3cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.775891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:31.790484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fae075fce4d94d679a8dc5f508a1c3cf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:31.797070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1052/2083] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:31 2022] DELETE /identity/v3/users/fae075fce4d94d679a8dc5f508a1c3cf => generated 0 bytes in 375 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:31.931007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:31.935000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:31.967010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c15916a5-8e8b-470e-bf6c-98af2bcf978f service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.049040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.049040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.049040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.049040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] PATH_INFO: `/v3/projects/7a96686d04894ed08b57af0320f43a88` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.050175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7a96686d04894ed08b57af0320f43a88)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.052791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.076495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.079253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.115369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aae46489-70e0-442a-9347-0c5955533631 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.121987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.126774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.127081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.127372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] PATH_INFO: `/v3/projects/7f23e9a6fbe44c69bb3e4ae47bfe93dc` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.128769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] RBAC: Authorizing `identity:delete_project(project_id=7f23e9a6fbe44c69bb3e4ae47bfe93dc)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.132826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.166011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7a96686d04894ed08b57af0320f43a88', 'request_id': 'req-0cd3c450-d0da-4b69-9c83-3d68207f46a2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.171678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7a96686d04894ed08b57af0320f43a88', 'request_id': 'req-0cd3c450-d0da-4b69-9c83-3d68207f46a2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.254916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7a96686d04894ed08b57af0320f43a88', 'request_id': 'req-0cd3c450-d0da-4b69-9c83-3d68207f46a2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.271277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7f23e9a6fbe44c69bb3e4ae47bfe93dc', 'request_id': 'req-25162a0c-475c-4cbc-bc06-a05b02029616'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.272006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] The token cache is being invalidate because project 7a96686d04894ed08b57af0320f43a88 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:32.272298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.273001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-0cd3c450-d0da-4b69-9c83-3d68207f46a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.274854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1032/2084] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:31 2022] DELETE /identity/v3/projects/7a96686d04894ed08b57af0320f43a88 => generated 0 bytes in 344 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:32.284362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011e91f-cc9e-4e0b-a133-465144c149fe None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.285004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011e91f-cc9e-4e0b-a133-465144c149fe None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.286163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4011e91f-cc9e-4e0b-a133-465144c149fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.286994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7f23e9a6fbe44c69bb3e4ae47bfe93dc', 'request_id': 'req-25162a0c-475c-4cbc-bc06-a05b02029616'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.299715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7f23e9a6fbe44c69bb3e4ae47bfe93dc', 'request_id': 'req-25162a0c-475c-4cbc-bc06-a05b02029616'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.321968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4011e91f-cc9e-4e0b-a133-465144c149fe None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:32.323845 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4011e91f-cc9e-4e0b-a133-465144c149fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.357372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] The token cache is being invalidate because project 7f23e9a6fbe44c69bb3e4ae47bfe93dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:32.362643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.365756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-25162a0c-475c-4cbc-bc06-a05b02029616 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:32.370122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:32.371757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1053/2085] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:32 2022] DELETE /identity/v3/projects/7f23e9a6fbe44c69bb3e4ae47bfe93dc => generated 0 bytes in 297 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:32.377167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.377936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.378226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.403910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1033/2086] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:32.409484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61e9f2-8772-4c9f-b6b4-8147c497f5a6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.409782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61e9f2-8772-4c9f-b6b4-8147c497f5a6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.410880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e61e9f2-8772-4c9f-b6b4-8147c497f5a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.427863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:32.429494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.432140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0e61e9f2-8772-4c9f-b6b4-8147c497f5a6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:32.433376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0e61e9f2-8772-4c9f-b6b4-8147c497f5a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.441407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1054/2087] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:32.443709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1034/2088] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:32.452868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.453162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64083a6-4cd0-4bca-9fbb-e688b2ef6667 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.453162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64083a6-4cd0-4bca-9fbb-e688b2ef6667 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.453162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d64083a6-4cd0-4bca-9fbb-e688b2ef6667 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.461319 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-026da290-d920-4e64-ba9d-4ad136041007 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.476964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d64083a6-4cd0-4bca-9fbb-e688b2ef6667 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:32.476964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d64083a6-4cd0-4bca-9fbb-e688b2ef6667 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.477259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.477259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.477259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.477259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.477611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:32.477611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.478678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.484015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1035/2089] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:32.486158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.486738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.487027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.498161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1055/2090] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:57:32.499176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.506347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:32.507847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.510980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13245c8f-b8fa-4fb8-af24-f5d2498799d9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.516224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.516922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.517366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.517651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.518405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:32.518722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.520373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.539629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1056/2091] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:57:32.539629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.548805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-269b5cfa-7854-4f55-93df-e01778701a35 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.560575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.561248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.561535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.562706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.562706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.564572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.571817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1036/2092] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:32.572697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.589758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2b9ce60-79c6-4913-9fe8-d7239a7218c7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.595842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.596536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.596832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.597136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.597838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.600025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.649885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1057/2093] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/projects => generated 365 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:32.649885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.649885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:32.723962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c4ce4d5-f023-483e-8afd-7cf2b4f6ee5f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.729097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.729746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.730026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.730309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.747766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1037/2094] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/projects => generated 371 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:32.748021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:32.748120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.829472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-620c8eec-fe5d-496e-a962-a56aff1581f5 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.834934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.836021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.836457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.837152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.838181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.841153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.850605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.855222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.868766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1058/2095] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:57:32 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 221 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:57:32.874638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.948413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1038/2096] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/users => generated 283 bytes in 199 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:32.950377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:32.965186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c2b9768-cd53-4e27-9198-78adb7c01882 service swift] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.969739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff64e86c-f957-4848-82bf-07f1dc162be9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:32.970689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.971420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.971710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.972008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.972826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.983039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.983167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:32.983167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:32.983167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:32.983167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:32.983477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:32.983477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:32.993541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1039/2097] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:32.998358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.009273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b06fd032-b1bd-485c-9127-ac8f7a6eb1bb admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.016009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.016705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.016989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.017263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.018005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.023857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.047246 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1040/2098] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:32 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:33.056598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.070198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1059/2099] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:32 2022] POST /identity/v3/users => generated 286 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:33.079983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.086696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c71a572e-1446-4027-bc33-b4175c8aef39 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.098998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adb46fb1-24b5-47fe-bb41-e9e775fd07bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.100052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46fb1-24b5-47fe-bb41-e9e775fd07bc admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.100409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46fb1-24b5-47fe-bb41-e9e775fd07bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.103286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adb46fb1-24b5-47fe-bb41-e9e775fd07bc admin admin] PATH_INFO: `/v3/projects/78b1be8bf03d44faa9341c4098035584/users/87e1f6329ed4439a9f32aa5f2acd8f0a/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.106969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-844fb422-d539-4a14-bf6b-a2b7b1466024 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.122236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.122934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.123292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.123548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.124411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.124535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46fb1-24b5-47fe-bb41-e9e775fd07bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=78b1be8bf03d44faa9341c4098035584, user_id=87e1f6329ed4439a9f32aa5f2acd8f0a, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.126056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.127846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adb46fb1-24b5-47fe-bb41-e9e775fd07bc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.133694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:33.160829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1060/2100] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:33.169904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.184878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd9d8f21-3edf-439d-a893-dcd4c36f06c3 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.193250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.194895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.195406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.196050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.197049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.199672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.216697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1041/2101] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:33 2022] PUT /identity/v3/projects/78b1be8bf03d44faa9341c4098035584/users/87e1f6329ed4439a9f32aa5f2acd8f0a/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:33.230976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.231546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.232086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.242906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1061/2102] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:33 2022] GET /identity/v3/roles => generated 1420 bytes in 73 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:33.248570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.269828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334df18d-932b-4014-9808-d48140b578d2 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.288904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] MFA Rules not processed for user `87e1f6329ed4439a9f32aa5f2acd8f0a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:33.290386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:33.377344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8e17753-f76d-462d-a83a-4761c1086dea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.378254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e17753-f76d-462d-a83a-4761c1086dea admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.378553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e17753-f76d-462d-a83a-4761c1086dea admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.378859 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8e17753-f76d-462d-a83a-4761c1086dea admin admin] PATH_INFO: `/v3/projects/b84ea0ffe5434521aa8965a31e20beac/users/2b0507a1874d450289debb3539cdf59d/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.399853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e17753-f76d-462d-a83a-4761c1086dea admin admin] RBAC: Authorizing `identity:create_grant(project_id=b84ea0ffe5434521aa8965a31e20beac, user_id=2b0507a1874d450289debb3539cdf59d, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.405275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8e17753-f76d-462d-a83a-4761c1086dea admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.465994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1042/2103] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 233 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:33.474976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.477212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:33.502746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1062/2104] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:33 2022] PUT /identity/v3/projects/b84ea0ffe5434521aa8965a31e20beac/users/2b0507a1874d450289debb3539cdf59d/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 255 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:33.513582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.519161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.519161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.529005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:33.571272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] MFA Rules not processed for user `2b0507a1874d450289debb3539cdf59d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:33.575504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:33.660819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-606a7533-a287-43a3-bf21-d683f5b2f688 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.669104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1063/2105] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:33 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 157 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:33.694415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.779412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.780834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.781816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.782380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.798397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bb8491-78e4-4113-bb37-9fc511392be3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:33.804069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:33.804787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:33.805124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:33.805372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:33.806603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.809830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.863623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:33.866605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:33.868335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:33.879402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1043/2106] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:33 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 405 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:33.955395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:33.959308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1064/2107] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:33 2022] POST /identity/v3/projects => generated 373 bytes in 266 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:33.978250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03c45fb6-31ac-4aa9-adcb-dc20fa125887 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:34.156615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:34.157613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:34.157996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:34.158372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:34.159496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:34.189168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:34.286948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1044/2108] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:33 2022] POST /identity/v3/users => generated 293 bytes in 332 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:34.292245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:34.306951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b8e74c-0abf-47b7-92ca-15072bb8854d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:34.324677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:34.325661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:34.327695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:34.327695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:34.327695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:34.339079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:34.370402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1065/2109] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 75 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:34.372525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:34.393656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57797d37-894b-43e5-a5c7-d8750fbcd29e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:34.399607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:34.400632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:34.401624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:34.401977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] PATH_INFO: `/v3/projects/faf35e66d6f6452ab1d014ecdc4260f0/users/3d3d583ddf0b4fbb84a092658c2513ef/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:34.420541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf35e66d6f6452ab1d014ecdc4260f0, user_id=3d3d583ddf0b4fbb84a092658c2513ef, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:34.423248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:34.477124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1045/2110] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:34 2022] PUT /identity/v3/projects/faf35e66d6f6452ab1d014ecdc4260f0/users/3d3d583ddf0b4fbb84a092658c2513ef/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:34.481310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:34.503163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-275edca4-01be-4853-ada9-6762b1080d2e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:34.649396 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf30597-bfc2-48a8-94b4-d075eca73e96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:34.650202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf30597-bfc2-48a8-94b4-d075eca73e96 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:34.650591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf30597-bfc2-48a8-94b4-d075eca73e96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:34.651620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf30597-bfc2-48a8-94b4-d075eca73e96 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:34.652338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf30597-bfc2-48a8-94b4-d075eca73e96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:34.654099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdf30597-bfc2-48a8-94b4-d075eca73e96 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:34.675903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1066/2111] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:34 2022] GET /identity/v3/roles => generated 1420 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:34.681588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:34.703012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29fb8520-4d6f-4969-a10d-ef094fc85c09 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:34.713942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:34.713942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:34.713942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:34.713942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] PATH_INFO: `/v3/projects/faf35e66d6f6452ab1d014ecdc4260f0/users/3d3d583ddf0b4fbb84a092658c2513ef/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:34.714339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=faf35e66d6f6452ab1d014ecdc4260f0, user_id=3d3d583ddf0b4fbb84a092658c2513ef, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:34.716971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:34.758079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1046/2112] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:34 2022] PUT /identity/v3/projects/faf35e66d6f6452ab1d014ecdc4260f0/users/3d3d583ddf0b4fbb84a092658c2513ef/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 77 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:34.774748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:34.775043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:34.775331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:34.785257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:34.828331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] MFA Rules not processed for user `3d3d583ddf0b4fbb84a092658c2513ef`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:34.832064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:34.986822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1067/2113] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:35.000919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:35.064424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78635363-53fb-4359-8cb7-49922ae399cd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:35.070071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:35.071046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:35.071329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:35.071660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:35.085083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:35.086857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:35.134667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1047/2114] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:34 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 127 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:35.376766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:35.387855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f93430-fe3d-44fa-bd89-468b42f9623d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:35.392740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-434fe055-dc9c-44d0-83f6-9dcc00decc07 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:35.394047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fe055-dc9c-44d0-83f6-9dcc00decc07 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:35.394842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fe055-dc9c-44d0-83f6-9dcc00decc07 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:35.395313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-434fe055-dc9c-44d0-83f6-9dcc00decc07 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:35.486136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fe055-dc9c-44d0-83f6-9dcc00decc07 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:35.489149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-434fe055-dc9c-44d0-83f6-9dcc00decc07 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:35.502401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1068/2115] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:35 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:45.108916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=72,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:45.496047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=60 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:56.171281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:56.173322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:56.270198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7eae434-1769-4881-b0d5-ed201643a406 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:56.271950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:56.273392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:56.273779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:56.274157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] PATH_INFO: `/v3/users/d18e9774031e49a2aaf213bb070758bc` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:56.275543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d18e9774031e49a2aaf213bb070758bc)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:56.278499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:56.511389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd18e9774031e49a2aaf213bb070758bc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:56.512212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd18e9774031e49a2aaf213bb070758bc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:56.620728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd18e9774031e49a2aaf213bb070758bc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:56.628764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd18e9774031e49a2aaf213bb070758bc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:56.641648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd18e9774031e49a2aaf213bb070758bc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:56.648475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1048/2116] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:56 2022] DELETE /identity/v3/users/d18e9774031e49a2aaf213bb070758bc => generated 0 bytes in 478 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:56.655863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:56.655863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:56.655863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:56.672686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:56.673890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:56.738486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1069/2117] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:56.749951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:56.751425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:56.819891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db8a8fb6-4441-49ef-9a39-e45589580547 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:56.824267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69c2240d-b6b6-450f-acee-836f1d64c50e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:56.824919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2240d-b6b6-450f-acee-836f1d64c50e service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:56.825204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2240d-b6b6-450f-acee-836f1d64c50e service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:56.825482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69c2240d-b6b6-450f-acee-836f1d64c50e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:56.835096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2240d-b6b6-450f-acee-836f1d64c50e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:56.836616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69c2240d-b6b6-450f-acee-836f1d64c50e service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:56.848043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1049/2118] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:57:57.380208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:57.382391 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:57.395566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76e4056b-293d-42e1-9d07-9d60a28709fb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:57.400390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:57.401248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:57.401534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:57.401811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] PATH_INFO: `/v3/projects/f431b3b8e5534e31ab70a9ab791e36e1` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:57.402988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f431b3b8e5534e31ab70a9ab791e36e1)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:57.406245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:57.556736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f431b3b8e5534e31ab70a9ab791e36e1', 'request_id': 'req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:57.565803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f431b3b8e5534e31ab70a9ab791e36e1', 'request_id': 'req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:57.572390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f431b3b8e5534e31ab70a9ab791e36e1', 'request_id': 'req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:57.653597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] The token cache is being invalidate because project f431b3b8e5534e31ab70a9ab791e36e1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:57.654019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:57.654670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3a928a5f-2877-4bf5-b74b-3bd5f26136c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:57.656370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1070/2119] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:57 2022] DELETE /identity/v3/projects/f431b3b8e5534e31ab70a9ab791e36e1 => generated 0 bytes in 277 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:57.661278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:57.661690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:57.662077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:57.720029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:57.721237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:57.742819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:57.785573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1050/2120] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:57.794836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:57.795131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:57.795417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:57.814698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:57.815970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:57.831546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1071/2121] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:57.836049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:57.846646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:57.847232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1694f4e0-4b09-4912-b868-c8fcc2f5754d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:57.848143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:57.851990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:57.852648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:57.852950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:57.853209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:57.854599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:57.854915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:57.856548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:57.861880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b38e061-7422-4dbf-90f6-eb8799825cd8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:57.870272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:57.871202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:57.871488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:57.871766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] PATH_INFO: `/v3/users/16a778d984d24024b8286acfb9ac48e3` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:57.872132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1051/2122] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:57:57.873336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] RBAC: Authorizing `identity:delete_user(user_id=16a778d984d24024b8286acfb9ac48e3)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:57.874197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:57.875280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:57.886403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e04e4cd5-1654-4585-bf8d-55f98d789ea7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:57.895627 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:57.897121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:57.897461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:57.897738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:57.898438 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:57.900673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:57.977101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1052/2123] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:57 2022] POST /identity/v3/projects => generated 373 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:57:57.981273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.038420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '16a778d984d24024b8286acfb9ac48e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:58.039141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '16a778d984d24024b8286acfb9ac48e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:58.045068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-573496f6-3000-4d6d-abfa-9299ea0ff7b1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.049951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.050624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.050921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.051224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.051958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.053912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:58.115210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '16a778d984d24024b8286acfb9ac48e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:58.122399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '16a778d984d24024b8286acfb9ac48e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:58.129426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=82,keystone:DELETE=16,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:58.134623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-367bf261-3161-4a83-94f0-2bf645e330bd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '16a778d984d24024b8286acfb9ac48e3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:58.141031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1072/2124] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:57:57 2022] DELETE /identity/v3/users/16a778d984d24024b8286acfb9ac48e3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:58.145686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.146002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.146290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.150130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1053/2125] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:57 2022] POST /identity/v3/users => generated 287 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:57:58.157174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.158822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:58.165306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:58.166559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:58.170692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-868a5394-107b-403f-92e1-af2fd937e2eb admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.233803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.234545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.234902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.235247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.236172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.240950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:58.247130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1073/2126] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:57:58.253598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1054/2127] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:57:58.254175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.255598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:58.257584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.268102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d296580-67be-4da9-9237-78e39c95dc16 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.273202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.273885 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.274203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.274488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.275225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.277463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:58.292251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1055/2128] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:58 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:57:58.298360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.298360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:58.303561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2708c780-e32c-46be-8a6c-952feb8b7c9c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.308946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.309911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.310289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.310794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] PATH_INFO: `/v3/projects/b91796b837b54453a01062ddeae35fc7/users/9431c21024b14dc685d797a53ea7c601/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.325562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78e3c9ba-6a79-466c-a691-c87f6cfd2f93 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.327963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b91796b837b54453a01062ddeae35fc7, user_id=9431c21024b14dc685d797a53ea7c601, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.330642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38a68d70-4d22-4838-b016-83f33403fdd0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.330794 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:58.332254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a68d70-4d22-4838-b016-83f33403fdd0 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.332628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a68d70-4d22-4838-b016-83f33403fdd0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.332980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38a68d70-4d22-4838-b016-83f33403fdd0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.345139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a68d70-4d22-4838-b016-83f33403fdd0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.346701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38a68d70-4d22-4838-b016-83f33403fdd0 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:58.357967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1074/2129] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:58.417283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1056/2130] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:58 2022] PUT /identity/v3/projects/b91796b837b54453a01062ddeae35fc7/users/9431c21024b14dc685d797a53ea7c601/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:58.427786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.428111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.428405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.456427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] MFA Rules not processed for user `9431c21024b14dc685d797a53ea7c601`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:58.457624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:58.521572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1075/2131] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:58 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:58.533867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.535575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:58.606518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9391d8-98a7-440b-9dfa-6c214b6e7252 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.610273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c146978-55be-4734-a91b-08b50b8c10cd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.610991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146978-55be-4734-a91b-08b50b8c10cd service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.611288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146978-55be-4734-a91b-08b50b8c10cd service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.611608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c146978-55be-4734-a91b-08b50b8c10cd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.625229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c146978-55be-4734-a91b-08b50b8c10cd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.626956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c146978-55be-4734-a91b-08b50b8c10cd service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:58.641745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1057/2132] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:57:58 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 108 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:57:58.888434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:58.889889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:58.920825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:58.967984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786f0cc5-99ee-48cc-9365-01613cbfad1c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:58.972836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:58.973701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:58.973985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:58.974263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] PATH_INFO: `/v3/projects/33d15c822d3541ca98ead932b369c0eb` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:58.975511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=33d15c822d3541ca98ead932b369c0eb)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:58.977528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.107753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '33d15c822d3541ca98ead932b369c0eb', 'request_id': 'req-c2636c89-7fc6-4daa-92c6-774be85249a1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:59.113860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '33d15c822d3541ca98ead932b369c0eb', 'request_id': 'req-c2636c89-7fc6-4daa-92c6-774be85249a1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:59.119260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '33d15c822d3541ca98ead932b369c0eb', 'request_id': 'req-c2636c89-7fc6-4daa-92c6-774be85249a1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:59.221291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] The token cache is being invalidate because project 33d15c822d3541ca98ead932b369c0eb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:57:59.221689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:59.224312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2636c89-7fc6-4daa-92c6-774be85249a1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:57:59.225964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1076/2133] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:57:58 2022] DELETE /identity/v3/projects/33d15c822d3541ca98ead932b369c0eb => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:57:59.236868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.237392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.237614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.255748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:59.257091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:59.317111 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1058/2134] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:59.324995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.325291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.325576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.347250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:57:59.348540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:57:59.362370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1077/2135] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:57:59.363052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:59.374216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baf267f1-43ee-43ea-b981-e0822c661a5d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:59.381188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:59.381863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.382145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.382414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.383296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:57:59.383609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:59.385276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.397613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1059/2136] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:57:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:57:59.402496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:59.412378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a0998d8-018f-4369-abc9-3bd3ff49026f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:59.417819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:59.418478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.418765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.419066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.419761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:59.421717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.538525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1078/2137] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:57:59 2022] POST /identity/v3/projects => generated 377 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:57:59.544359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:59.554509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2eca559-d643-4fa0-afb8-f8e14f61ed5b admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:59.660680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:59.661992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.662352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.662706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.663787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:59.665784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.775583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:57:59.784471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1060/2138] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:57:59 2022] POST /identity/v3/users => generated 289 bytes in 242 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:57:59.788764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:59.799427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b7e692-68a8-4cb2-b3e0-918d0360d1af admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:59.804147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:59.804804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.805091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.805371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.806097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:59.807724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.818025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1079/2139] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:59.821820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:59.832421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-151132be-09fd-4a28-9b07-e4a865102105 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:59.837306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6087acda-860e-4a34-99c0-4c0bbf2c1ed0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:59.838046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087acda-860e-4a34-99c0-4c0bbf2c1ed0 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.838985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087acda-860e-4a34-99c0-4c0bbf2c1ed0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.839263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6087acda-860e-4a34-99c0-4c0bbf2c1ed0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.841420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087acda-860e-4a34-99c0-4c0bbf2c1ed0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:59.841783 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6087acda-860e-4a34-99c0-4c0bbf2c1ed0 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.852642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1061/2140] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:57:59 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:57:59.856747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:57:59.869036 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-423d01dd-ea9a-4b81-a42f-ac26102b59df admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:57:59.878617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:57:59.879613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.879927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.880204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] PATH_INFO: `/v3/projects/317f3f13adee4c7daa20d3f6b06a2d8b/users/d85a031a17ab4b09b846f355f7e1476d/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:57:59.896500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] RBAC: Authorizing `identity:create_grant(project_id=317f3f13adee4c7daa20d3f6b06a2d8b, user_id=d85a031a17ab4b09b846f355f7e1476d, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:57:59.899243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:57:59.978574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1080/2141] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:57:59 2022] PUT /identity/v3/projects/317f3f13adee4c7daa20d3f6b06a2d8b/users/d85a031a17ab4b09b846f355f7e1476d/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:57:59.989765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:57:59.990062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:57:59.990345 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:00.016656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] MFA Rules not processed for user `d85a031a17ab4b09b846f355f7e1476d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:00.018028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:00.081569 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1062/2142] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:57:59 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:00.092810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:00.094824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:00.153877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:00.159266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fe717ac-8d36-46dd-b1b7-4ad0cad99f89 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:00.163007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:00.163669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:00.163990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:00.164273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:00.173179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:00.174514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:00.184529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1081/2143] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:00 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:10.078646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=38 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:10.326500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:14.944430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:14.945840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:15.005939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ea0cc7a-d902-4f83-9301-66c344cdc075 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:15.009586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:15.010232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:15.010525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:15.010802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:15.012632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:15.078147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:15.079730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d6bd14e-9372-41da-8a84-3c01b0c6027d service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:15.090315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1063/2144] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:14 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:21.236570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:21.239629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:21.513412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e41e667-5cc8-499b-83e5-10b046f89c6b service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:21.519505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:21.520721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:21.521226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:21.521679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] PATH_INFO: `/v3/users/9431c21024b14dc685d797a53ea7c601` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:21.523145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] RBAC: Authorizing `identity:delete_user(user_id=9431c21024b14dc685d797a53ea7c601)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:21.526318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:21.724354 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9431c21024b14dc685d797a53ea7c601'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:21.725172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9431c21024b14dc685d797a53ea7c601'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:21.867151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9431c21024b14dc685d797a53ea7c601'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:21.882450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9431c21024b14dc685d797a53ea7c601'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:21.908563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9431c21024b14dc685d797a53ea7c601'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:21.918675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1082/2145] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:21 2022] DELETE /identity/v3/users/9431c21024b14dc685d797a53ea7c601 => generated 0 bytes in 683 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:21.923497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:21.924048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:21.925661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:21.960078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:21.963524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:22.026340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1064/2146] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:21 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:22.035816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:22.037254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:22.060640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:22.133795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91245262-ade1-46b4-9d5e-9b42a9c92d3d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:22.137440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1095aff-f881-4017-9d70-e450a75984cc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:22.138078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1095aff-f881-4017-9d70-e450a75984cc service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:22.138364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1095aff-f881-4017-9d70-e450a75984cc service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:22.138638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1095aff-f881-4017-9d70-e450a75984cc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:22.147398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1095aff-f881-4017-9d70-e450a75984cc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:22.148883 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1095aff-f881-4017-9d70-e450a75984cc service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:22.158118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1083/2147] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:22 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:22.657092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:22.658521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:22.671240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4cd0e706-353f-4293-b61e-271da1e515b5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:22.688507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:22.690888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:22.691279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:22.691452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] PATH_INFO: `/v3/projects/b91796b837b54453a01062ddeae35fc7` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:22.692720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] RBAC: Authorizing `identity:delete_project(project_id=b91796b837b54453a01062ddeae35fc7)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:22.694640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:22.796598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:22.803165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b91796b837b54453a01062ddeae35fc7', 'request_id': 'req-bfb94562-1fc6-4d8c-a36c-6908452c638f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:22.808701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b91796b837b54453a01062ddeae35fc7', 'request_id': 'req-bfb94562-1fc6-4d8c-a36c-6908452c638f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:22.909455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b91796b837b54453a01062ddeae35fc7', 'request_id': 'req-bfb94562-1fc6-4d8c-a36c-6908452c638f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:22.921506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] The token cache is being invalidate because project b91796b837b54453a01062ddeae35fc7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:22.921804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:22.922457 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:22.924030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1065/2148] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:22 2022] DELETE /identity/v3/projects/b91796b837b54453a01062ddeae35fc7 => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:32.247688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=34 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:33.086461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=10,keystone:DELETE=4,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:36.633175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d374-b10a-4b21-9cdd-a48f4727d7fa None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:36.633716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d374-b10a-4b21-9cdd-a48f4727d7fa None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:36.633881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd12d374-b10a-4b21-9cdd-a48f4727d7fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:36.654301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fd12d374-b10a-4b21-9cdd-a48f4727d7fa None None] MFA Rules not processed for user `6cdd0a7f3f12414ea85ebffc9d531773`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:36.655619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fd12d374-b10a-4b21-9cdd-a48f4727d7fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:36.717728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1084/2149] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:36 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:36.983579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:36.985204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:37.065533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb94562-1fc6-4d8c-a36c-6908452c638f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:37.070729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:37.071633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:37.071938 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:37.072217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] PATH_INFO: `/v3/users/d85a031a17ab4b09b846f355f7e1476d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:37.073535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d85a031a17ab4b09b846f355f7e1476d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:37.075716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:37.113163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:37.113556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:37.113855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:37.147469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:37.148681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:37.221732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1085/2150] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:58:37 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:58:37.283455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:37.285588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:37.293826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd85a031a17ab4b09b846f355f7e1476d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:37.294712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd85a031a17ab4b09b846f355f7e1476d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:37.375257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99388d96-e803-4b34-aee6-5c838cd3a5e5 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:37.382416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a7722a8-da53-4629-82ea-4043f65214bd service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:37.383090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7722a8-da53-4629-82ea-4043f65214bd service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:37.383386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7722a8-da53-4629-82ea-4043f65214bd service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:37.383666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a7722a8-da53-4629-82ea-4043f65214bd service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:37.396931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7722a8-da53-4629-82ea-4043f65214bd service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:37.398886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a7722a8-da53-4629-82ea-4043f65214bd service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:37.412235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1086/2151] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:58:37 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:58:37.415856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd85a031a17ab4b09b846f355f7e1476d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:37.425658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd85a031a17ab4b09b846f355f7e1476d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:37.453275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd85a031a17ab4b09b846f355f7e1476d'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:37.463595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1066/2152] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:36 2022] DELETE /identity/v3/users/d85a031a17ab4b09b846f355f7e1476d => generated 0 bytes in 480 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:37.468770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:37.469096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:37.470031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:37.497600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:37.499035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:37.576383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:37.632846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1087/2153] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:58:37.643182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:37.644995 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:37.752359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55458d33-f410-4e6d-b971-a921616ea1d5 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:37.756484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8b46b5f-e01b-442c-8ebf-6859c79fd2f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:37.757149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b46b5f-e01b-442c-8ebf-6859c79fd2f7 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:37.757434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b46b5f-e01b-442c-8ebf-6859c79fd2f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:37.757728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8b46b5f-e01b-442c-8ebf-6859c79fd2f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:37.769153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b46b5f-e01b-442c-8ebf-6859c79fd2f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:37.777539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8b46b5f-e01b-442c-8ebf-6859c79fd2f7 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:37.789300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1067/2154] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 147 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:38.310306 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:38.311832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:38.333957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692cf892-d82b-4cc1-afa9-d142e1f3a04e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:38.348033 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:38.348951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:38.349249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:38.349540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] PATH_INFO: `/v3/projects/317f3f13adee4c7daa20d3f6b06a2d8b` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:38.350891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=317f3f13adee4c7daa20d3f6b06a2d8b)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:38.352909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:38.473393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '317f3f13adee4c7daa20d3f6b06a2d8b', 'request_id': 'req-b7f850c7-a345-4abd-877b-67a608bee7a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:38.480982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '317f3f13adee4c7daa20d3f6b06a2d8b', 'request_id': 'req-b7f850c7-a345-4abd-877b-67a608bee7a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:38.486865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '317f3f13adee4c7daa20d3f6b06a2d8b', 'request_id': 'req-b7f850c7-a345-4abd-877b-67a608bee7a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:38.573360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] The token cache is being invalidate because project 317f3f13adee4c7daa20d3f6b06a2d8b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:38.573360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:38.574441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b7f850c7-a345-4abd-877b-67a608bee7a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:38.576273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1088/2155] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:38 2022] DELETE /identity/v3/projects/317f3f13adee4c7daa20d3f6b06a2d8b => generated 0 bytes in 267 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:38.583521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:38.584400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:38.584400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:38.630552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:38.683582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:38.692601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:38.797291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1068/2156] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:38.799813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:38.802872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:38.803492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.058696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:39.058696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:39.058696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1089/2157] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:39.064904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.064904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c921aaf1-d405-4bbf-8cb5-7d1d7e71dc4e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.077469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.078162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.078456 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.079299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.080074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:58:39.080387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.082326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:39.122779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1069/2158] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:58:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:58:39.127774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.139059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9ab2166-5462-416b-a3b8-502e6c8e0c58 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.154869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.154869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.154869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.154869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.155239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.155239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:39.460590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1090/2159] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:58:39 2022] POST /identity/v3/projects => generated 373 bytes in 333 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:39.465022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.478103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d674806d-b106-4e38-ad17-d2fce582e68b admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.582917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.583916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.584502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.587747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.588439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.591072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:39.737504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1070/2160] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:58:39 2022] POST /identity/v3/users => generated 287 bytes in 273 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:39.742009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.754602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6235e76f-26a9-4362-b32b-c6bd545c0bb0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.762039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.762039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.762039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.762039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.763578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.765269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:39.787309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1091/2161] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:58:39.793747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.821198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c395b328-196a-4cf6-ad76-6f8132cf4dd3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.841760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.842707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.842994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.846284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.846284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.846284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:39.848057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.850007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:39.872980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b5f346-8482-4838-a47f-63efae39b3a4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.877969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.878913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.879174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.879447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] PATH_INFO: `/v3/users/2b0507a1874d450289debb3539cdf59d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.880725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2b0507a1874d450289debb3539cdf59d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.881825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1071/2162] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:39 2022] GET /identity/v3/roles => generated 1420 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:39.882685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:39.882830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:39.914824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b676431-f2da-4489-9da8-aa166a7ef28d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:39.925041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d03baa67-f173-4910-b911-04e8c6e78826 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:39.926338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03baa67-f173-4910-b911-04e8c6e78826 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:39.927098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03baa67-f173-4910-b911-04e8c6e78826 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:39.929467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d03baa67-f173-4910-b911-04e8c6e78826 admin admin] PATH_INFO: `/v3/projects/1c8858f51824444782c874f15da7b16b/users/9082354c3ab249f885e129b86799e508/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:39.959745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03baa67-f173-4910-b911-04e8c6e78826 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1c8858f51824444782c874f15da7b16b, user_id=9082354c3ab249f885e129b86799e508, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:39.965739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d03baa67-f173-4910-b911-04e8c6e78826 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:40.098861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1072/2163] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:39 2022] PUT /identity/v3/projects/1c8858f51824444782c874f15da7b16b/users/9082354c3ab249f885e129b86799e508/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 216 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:40.103557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:40.113701 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:40.120029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:40.130499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2b0507a1874d450289debb3539cdf59d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:40.130810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2b0507a1874d450289debb3539cdf59d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:40.146734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:40.148149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:40.219077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2b0507a1874d450289debb3539cdf59d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:40.231499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2b0507a1874d450289debb3539cdf59d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:40.242340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=81,keystone:DELETE=16,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:40.256944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e3df6ee5-87fa-43db-90ee-82c29c76cb96 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2b0507a1874d450289debb3539cdf59d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:40.262508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:40.270047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1092/2164] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:39 2022] DELETE /identity/v3/users/2b0507a1874d450289debb3539cdf59d => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:40.285371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:40.285774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:40.286154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:40.322464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:40.327913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:40.339318 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1073/2165] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 236 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:40.348578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:40.350626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:40.464979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1093/2166] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:58:40.468729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32f8ccdb-63c2-4de7-bb9e-8212490197e0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:40.471975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:40.541432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9e4eaf2-ea27-45cb-a393-864cc02b5158 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:40.548018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:40.548699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:40.549919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:40.550276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:40.562795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:40.564556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:40.599271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:40.599271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:40.599271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:40.599271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:40.600148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1094/2167] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:58:40.609360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:40.612338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:40.623699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1074/2168] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 276 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:58:41.458804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:41.461419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:41.495736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8fbfc30-0502-472d-af60-ed306cb37c95 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:41.507584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:41.508526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:41.508817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:41.509521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] PATH_INFO: `/v3/projects/b84ea0ffe5434521aa8965a31e20beac` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:41.510841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b84ea0ffe5434521aa8965a31e20beac)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:41.513310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:41.689796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b84ea0ffe5434521aa8965a31e20beac', 'request_id': 'req-8b97fad5-2729-4709-b082-0c2f8919b7b3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:41.700105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b84ea0ffe5434521aa8965a31e20beac', 'request_id': 'req-8b97fad5-2729-4709-b082-0c2f8919b7b3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:41.713476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b84ea0ffe5434521aa8965a31e20beac', 'request_id': 'req-8b97fad5-2729-4709-b082-0c2f8919b7b3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:41.886947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] The token cache is being invalidate because project b84ea0ffe5434521aa8965a31e20beac was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:41.888280 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:41.889360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:41.892255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1095/2169] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:41 2022] DELETE /identity/v3/projects/b84ea0ffe5434521aa8965a31e20beac => generated 0 bytes in 434 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:41.896310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:41.912028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcde9777-df5d-4ade-953a-5d57ebaf7c6b service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:42.009987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:42.013113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:42.013113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:42.013113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] PATH_INFO: `/v3/users/3d3d583ddf0b4fbb84a092658c2513ef` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:42.013525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d3d583ddf0b4fbb84a092658c2513ef)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:42.016007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:42.122405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:42.124965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:42.156180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b97fad5-2729-4709-b082-0c2f8919b7b3 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:42.166038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:42.167961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:42.169098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:42.169603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] PATH_INFO: `/v3/users/87e1f6329ed4439a9f32aa5f2acd8f0a` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:42.172456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] RBAC: Authorizing `identity:delete_user(user_id=87e1f6329ed4439a9f32aa5f2acd8f0a)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:42.175538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:42.187943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:42.216696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:42.227133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d3d583ddf0b4fbb84a092658c2513ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.227860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d3d583ddf0b4fbb84a092658c2513ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.355678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d3d583ddf0b4fbb84a092658c2513ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.375311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d3d583ddf0b4fbb84a092658c2513ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.392574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-780ee130-ebfb-487b-aca0-c34e7ba0eb34 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d3d583ddf0b4fbb84a092658c2513ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.401941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1075/2170] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:41 2022] DELETE /identity/v3/users/3d3d583ddf0b4fbb84a092658c2513ef => generated 0 bytes in 506 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:42.480393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87e1f6329ed4439a9f32aa5f2acd8f0a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.482535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87e1f6329ed4439a9f32aa5f2acd8f0a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.596762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87e1f6329ed4439a9f32aa5f2acd8f0a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.608467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87e1f6329ed4439a9f32aa5f2acd8f0a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.626683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87e1f6329ed4439a9f32aa5f2acd8f0a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:42.634761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1096/2171] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:42 2022] DELETE /identity/v3/users/87e1f6329ed4439a9f32aa5f2acd8f0a => generated 0 bytes in 514 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:42.639515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:42.639815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:42.640201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:42.663330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:42.666035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:42.808151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1076/2172] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:42.869113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:42.872664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:42.988869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c2c4951-4847-4b6f-a9f6-70f4e1fed6cd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:42.993643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0193b989-4c1f-4865-b0f7-ae84a5f8c052 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:42.997945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193b989-4c1f-4865-b0f7-ae84a5f8c052 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:42.997945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193b989-4c1f-4865-b0f7-ae84a5f8c052 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:42.997945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0193b989-4c1f-4865-b0f7-ae84a5f8c052 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.009756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193b989-4c1f-4865-b0f7-ae84a5f8c052 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:43.011837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0193b989-4c1f-4865-b0f7-ae84a5f8c052 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:43.043084 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1097/2173] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:43.063287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:43.064919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:43.089326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81d7eef-eb84-43fc-afa7-2ea93731c8f4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:43.095158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:43.112125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:43.112502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:43.112790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] PATH_INFO: `/v3/projects/faf35e66d6f6452ab1d014ecdc4260f0` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.114355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] RBAC: Authorizing `identity:delete_project(project_id=faf35e66d6f6452ab1d014ecdc4260f0)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:43.118559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:43.285662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'faf35e66d6f6452ab1d014ecdc4260f0', 'request_id': 'req-79a37fa1-5190-4878-8e4f-5f22212d2406'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.293340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'faf35e66d6f6452ab1d014ecdc4260f0', 'request_id': 'req-79a37fa1-5190-4878-8e4f-5f22212d2406'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.394710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'faf35e66d6f6452ab1d014ecdc4260f0', 'request_id': 'req-79a37fa1-5190-4878-8e4f-5f22212d2406'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.429703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] The token cache is being invalidate because project faf35e66d6f6452ab1d014ecdc4260f0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:43.430054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.431522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-79a37fa1-5190-4878-8e4f-5f22212d2406 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.443967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3f7cd-ef62-48d5-bf90-92fd3b5ef6dd None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:43.443967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3f7cd-ef62-48d5-bf90-92fd3b5ef6dd None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:43.443967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ea3f7cd-ef62-48d5-bf90-92fd3b5ef6dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.446819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1077/2174] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:43 2022] DELETE /identity/v3/projects/faf35e66d6f6452ab1d014ecdc4260f0 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:43.475605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4ea3f7cd-ef62-48d5-bf90-92fd3b5ef6dd None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:43.478343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4ea3f7cd-ef62-48d5-bf90-92fd3b5ef6dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:43.559385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:43.564930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:43.565985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.615096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1098/2175] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:43.627154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:43.627788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:43.628359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.653102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] MFA Rules not processed for user `9082354c3ab249f885e129b86799e508`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:43.654363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:43.716584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:43.717832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:43.738127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=87,keystone:DELETE=11,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:43.743950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1099/2176] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:43.748113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:43.748577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:43.762568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a70d6cf0-bfa2-4c1b-9729-4fa3fb1d5970 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:43.774412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:43.777578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:43.778298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:43.778881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] PATH_INFO: `/v3/projects/78b1be8bf03d44faa9341c4098035584` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.780787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=78b1be8bf03d44faa9341c4098035584)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:43.783221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:43.826788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:43.842370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1078/2177] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:43 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 290 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:43.846216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:43.874365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d0b450a-37d8-40b0-8b15-5c19f0e28ef1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:43.888662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:43.889650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:43.890125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:43.890607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:43.891552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:58:43.892770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:43.900929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:43.918974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1079/2178] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:58:43 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 76 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:58:43.920471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:43.923417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:43.975411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '78b1be8bf03d44faa9341c4098035584', 'request_id': 'req-4236a68a-0827-4b9c-b67c-1a5ef79294f3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.985052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '78b1be8bf03d44faa9341c4098035584', 'request_id': 'req-4236a68a-0827-4b9c-b67c-1a5ef79294f3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:43.999614 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '78b1be8bf03d44faa9341c4098035584', 'request_id': 'req-4236a68a-0827-4b9c-b67c-1a5ef79294f3'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:44.032940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc999fdf-14fa-4aff-afb4-0fd4bea0e678 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.102712 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f9aa761-8155-4bd6-93cd-6d538f67010e service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.104378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9aa761-8155-4bd6-93cd-6d538f67010e service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.105437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9aa761-8155-4bd6-93cd-6d538f67010e service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.105931 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f9aa761-8155-4bd6-93cd-6d538f67010e service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.118181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] The token cache is being invalidate because project 78b1be8bf03d44faa9341c4098035584 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:44.118607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:44.124755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:44.126121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1100/2179] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:43 2022] DELETE /identity/v3/projects/78b1be8bf03d44faa9341c4098035584 => generated 0 bytes in 381 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:44.127336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.128957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:44.259689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4236a68a-0827-4b9c-b67c-1a5ef79294f3 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.272756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.276656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.278336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.279676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.280556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.293924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.295330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9aa761-8155-4bd6-93cd-6d538f67010e service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.297762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f9aa761-8155-4bd6-93cd-6d538f67010e service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.337645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1080/2180] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:43 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 418 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:44.343242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd15fe-fe67-4bd3-a245-4f92c2f55b4b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.343798 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd15fe-fe67-4bd3-a245-4f92c2f55b4b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.344094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd15fe-fe67-4bd3-a245-4f92c2f55b4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.438400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0fdd15fe-fe67-4bd3-a245-4f92c2f55b4b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:44.439832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0fdd15fe-fe67-4bd3-a245-4f92c2f55b4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:44.464542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1101/2181] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:58:44 2022] POST /identity/v3/projects => generated 365 bytes in 338 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:44.466132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.500056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1081/2182] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 155 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:44.502823 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.503127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.503413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.579132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:44.580557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:44.590465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9011cf90-5ba9-40cb-86db-d80da1b16d4d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.606232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1082/2183] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:44.606436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.606436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.606436 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.611486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.611604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.611693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.612338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.619853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-027305df-70bb-4c6e-8c05-c85b96c1d885 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.631521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.634449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.634744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.635141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.636062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:58:44.636455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.638167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.668704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1083/2184] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:58:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 62 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:58:44.672918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.696877 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3f6b397-b5db-42f1-9862-1558740d4d69 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.717085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.718115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.718762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.719141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.720310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.722346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.774235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1102/2185] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:58:44 2022] POST /identity/v3/users => generated 283 bytes in 309 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:44.778952 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.791017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad10e29-c359-4453-a163-928477561565 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.808706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.809414 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.809761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.810074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.811647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.813535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.829180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1103/2186] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:44.836142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.853936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27a39297-c428-4c90-b98a-90c04ab174ca admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.859274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.861543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.861660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1084/2187] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:58:44 2022] POST /identity/v3/projects => generated 375 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:44.861878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.862333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.864813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.865671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:44.868601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.879548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1104/2188] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:44 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:44.883492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:44.896662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:44.900263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f304f3-e944-4607-9a67-80a0798607cc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.981154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf3fc662-1e24-471d-a995-4f584e34465e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:44.986752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:44.988555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:44.988555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:44.988555 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:44.989369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:44.991629 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.009119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca83910-5ba4-4eef-bcaa-76b10d77a623 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.010066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83910-5ba4-4eef-bcaa-76b10d77a623 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.010366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83910-5ba4-4eef-bcaa-76b10d77a623 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.010654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca83910-5ba4-4eef-bcaa-76b10d77a623 admin admin] PATH_INFO: `/v3/projects/c4d412c65568489fb45ac51b62b407d1/users/5112ab12f11e4ae7b5ea35e243fa0e72/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.028080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca83910-5ba4-4eef-bcaa-76b10d77a623 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c4d412c65568489fb45ac51b62b407d1, user_id=5112ab12f11e4ae7b5ea35e243fa0e72, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.031540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca83910-5ba4-4eef-bcaa-76b10d77a623 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.103287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1085/2189] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:58:44 2022] POST /identity/v3/users => generated 288 bytes in 232 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:45.105910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.128471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3759a626-f15d-41c1-8a18-aa2369e09d16 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.137509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.138634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.139090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.139557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.140490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.143249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.152801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1105/2190] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:44 2022] PUT /identity/v3/projects/c4d412c65568489fb45ac51b62b407d1/users/5112ab12f11e4ae7b5ea35e243fa0e72/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:58:45.163337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1086/2191] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:45.171658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.179221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.179813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.180297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.185433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:45.193600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb4270e-e4a4-431a-b907-0a41c07a5f41 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.263671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] MFA Rules not processed for user `5112ab12f11e4ae7b5ea35e243fa0e72`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:45.266207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:45.314328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.316215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.317216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.317787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.318745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.321009 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.337753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1087/2192] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:45.349216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.374229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95963e15-cf66-4d0f-ad66-281ec077d988 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.390130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e99cb6-43b2-4c8c-96bd-d0934f1ade41 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.391138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e99cb6-43b2-4c8c-96bd-d0934f1ade41 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.391552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e99cb6-43b2-4c8c-96bd-d0934f1ade41 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.391734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1106/2193] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:45 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 219 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:45.392640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.393847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e99cb6-43b2-4c8c-96bd-d0934f1ade41 admin admin] PATH_INFO: `/v3/projects/c6602646a8734415a2cd5c138ec99b29/users/e32560e5ffa640d0b505834f090bd330/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.413983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e99cb6-43b2-4c8c-96bd-d0934f1ade41 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c6602646a8734415a2cd5c138ec99b29, user_id=e32560e5ffa640d0b505834f090bd330, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.418082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e99cb6-43b2-4c8c-96bd-d0934f1ade41 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.425386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3da07fd-f568-478e-8662-79e553f9f990 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.436474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.437279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.437705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.438169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.439404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.441887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.544501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1088/2194] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:45 2022] PUT /identity/v3/projects/c6602646a8734415a2cd5c138ec99b29/users/e32560e5ffa640d0b505834f090bd330/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:45.555708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.556033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.556315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.559464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1107/2195] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:58:45 2022] POST /identity/v3/projects => generated 365 bytes in 167 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:45.564825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.614746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] MFA Rules not processed for user `e32560e5ffa640d0b505834f090bd330`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:45.616023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:45.639887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:45.726174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41199a24-f87c-4f9a-9b13-9f10700b748c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.735211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.735456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.735930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.736446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.737816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.740452 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.746743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1089/2196] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:45 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:58:45.762340 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.765222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:45.859044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4006bc2-cd61-4693-b9fa-a357a89d2c88 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.864481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.865249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.865604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.865951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.883273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1108/2197] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:58:45 2022] POST /identity/v3/users => generated 289 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:45.888461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.899990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.899990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.906118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1090/2198] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:45 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:45.916892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad73ceeb-05f7-4079-8063-da2859cac04d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:45.939207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:45.939921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:45.940211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:45.940490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:45.941227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:45.942915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:45.965477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1109/2199] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:45 2022] GET /identity/v3/roles => generated 1420 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:45.982933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:45.994485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdd4559b-e072-4984-934a-973ccbc2c8ea service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:46.000049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:46.000946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:46.001229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:46.001507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] PATH_INFO: `/v3/projects/055d13f46574420283883011c8119db3/users/ff794327fe1d4cbb9c769076ffe73dd5/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:46.016756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] RBAC: Authorizing `identity:create_grant(project_id=055d13f46574420283883011c8119db3, user_id=ff794327fe1d4cbb9c769076ffe73dd5, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:46.019411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:46.386344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1091/2200] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:45 2022] PUT /identity/v3/projects/055d13f46574420283883011c8119db3/users/ff794327fe1d4cbb9c769076ffe73dd5/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 404 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:58:46.390426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:46.400924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0115684b-0432-42d7-b212-3a9a47bba0d4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:46.505534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ce6aaff-344f-493e-b29d-5794c49ed5f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:46.506246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6aaff-344f-493e-b29d-5794c49ed5f5 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:46.506534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6aaff-344f-493e-b29d-5794c49ed5f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:46.506810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ce6aaff-344f-493e-b29d-5794c49ed5f5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:46.507541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6aaff-344f-493e-b29d-5794c49ed5f5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:46.509316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ce6aaff-344f-493e-b29d-5794c49ed5f5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:46.523774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1110/2201] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:46 2022] GET /identity/v3/roles => generated 1420 bytes in 134 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:46.528978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:46.541717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad9e1e9-da39-47e1-b67a-ad86b44e775d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:46.548306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:46.557264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:46.557264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:46.557264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] PATH_INFO: `/v3/projects/055d13f46574420283883011c8119db3/users/ff794327fe1d4cbb9c769076ffe73dd5/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:46.559241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] RBAC: Authorizing `identity:create_grant(project_id=055d13f46574420283883011c8119db3, user_id=ff794327fe1d4cbb9c769076ffe73dd5, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:46.562442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:46.583354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1092/2202] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:46 2022] PUT /identity/v3/projects/055d13f46574420283883011c8119db3/users/ff794327fe1d4cbb9c769076ffe73dd5/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:46.594650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e926902-eea1-421f-9443-28e5da7523a6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:46.594953 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e926902-eea1-421f-9443-28e5da7523a6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:46.595236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e926902-eea1-421f-9443-28e5da7523a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:46.605525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:46.633043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9e926902-eea1-421f-9443-28e5da7523a6 None None] MFA Rules not processed for user `ff794327fe1d4cbb9c769076ffe73dd5`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:46.634603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9e926902-eea1-421f-9443-28e5da7523a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:46.801931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1111/2203] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:46 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 204 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:46.810539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:46.812779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:46.906020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ec36a89-4e76-4ebd-b5dd-351c66e53277 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:46.914194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e647403-f5ba-465c-9efc-29f35e4ae702 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:46.914959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647403-f5ba-465c-9efc-29f35e4ae702 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:46.915355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647403-f5ba-465c-9efc-29f35e4ae702 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:46.915680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e647403-f5ba-465c-9efc-29f35e4ae702 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:46.927526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e647403-f5ba-465c-9efc-29f35e4ae702 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:46.929242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e647403-f5ba-465c-9efc-29f35e4ae702 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:46.941260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1093/2204] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:46 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:47.392460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:47.392912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:47.393102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:47.413855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] MFA Rules not processed for user `9082354c3ab249f885e129b86799e508`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:47.415173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:47.497978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:47.511977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1112/2205] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:47 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:48.120077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fd8f1-2e3f-4949-893a-da9624972367 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:48.120077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fd8f1-2e3f-4949-893a-da9624972367 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:48.120077 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-128fd8f1-2e3f-4949-893a-da9624972367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:48.124384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:48.156560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-128fd8f1-2e3f-4949-893a-da9624972367 None None] MFA Rules not processed for user `e32560e5ffa640d0b505834f090bd330`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:48.158394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-128fd8f1-2e3f-4949-893a-da9624972367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:48.191160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:48.289700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1094/2206] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:48 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:48.339929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8287915-ad03-452c-bb22-7737c9721818 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:48.364081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:48.364903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:48.365184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:48.365469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] PATH_INFO: `/v3/users/5112ab12f11e4ae7b5ea35e243fa0e72` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:48.371035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5112ab12f11e4ae7b5ea35e243fa0e72)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:48.382120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:48.657109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5112ab12f11e4ae7b5ea35e243fa0e72'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:48.658188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5112ab12f11e4ae7b5ea35e243fa0e72'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:48.791074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5112ab12f11e4ae7b5ea35e243fa0e72'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:48.800681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5112ab12f11e4ae7b5ea35e243fa0e72'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:48.838141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5112ab12f11e4ae7b5ea35e243fa0e72'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:48.845968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1113/2207] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:48 2022] DELETE /identity/v3/users/5112ab12f11e4ae7b5ea35e243fa0e72 => generated 0 bytes in 722 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:48.850692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:48.851250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:48.851546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:48.893260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:48.893483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:48.990019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1095/2208] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 140 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:49.003602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:49.005254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:49.108404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93f318a8-b48f-455f-ba02-76682decc965 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:49.114680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:49.115491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:49.115913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:49.116445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:49.130664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:49.132518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:49.157131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1114/2209] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 153 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:49.800160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:49.800160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:49.827448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5e8c8b-f40b-4ef1-80c8-ab1fad7a0bd8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:49.844008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:49.845361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:49.845853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:49.846396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] PATH_INFO: `/v3/projects/c4d412c65568489fb45ac51b62b407d1` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:49.847908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c4d412c65568489fb45ac51b62b407d1)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:49.850010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:50.013593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c4d412c65568489fb45ac51b62b407d1', 'request_id': 'req-b957640b-faac-49e9-9c04-7f10314c42a7'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:50.028288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c4d412c65568489fb45ac51b62b407d1', 'request_id': 'req-b957640b-faac-49e9-9c04-7f10314c42a7'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:50.122518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c4d412c65568489fb45ac51b62b407d1', 'request_id': 'req-b957640b-faac-49e9-9c04-7f10314c42a7'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:50.159825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] The token cache is being invalidate because project c4d412c65568489fb45ac51b62b407d1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:50.160227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:50.160907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:50.167623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:50.183745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1096/2210] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:49 2022] DELETE /identity/v3/projects/c4d412c65568489fb45ac51b62b407d1 => generated 0 bytes in 366 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:50.198021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-193b07b2-53db-4ebf-a60f-df3ea3d42458 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:50.277782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:50.337565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:50.338453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:50.339265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:50.339549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] PATH_INFO: `/v3/users/ff794327fe1d4cbb9c769076ffe73dd5` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:50.340885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ff794327fe1d4cbb9c769076ffe73dd5)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:50.342902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:50.940465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ff794327fe1d4cbb9c769076ffe73dd5'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:50.941999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ff794327fe1d4cbb9c769076ffe73dd5'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:51.060508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ff794327fe1d4cbb9c769076ffe73dd5'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:51.071838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ff794327fe1d4cbb9c769076ffe73dd5'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:51.123151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-eec62cc7-ae65-40e1-a222-5644eb47f494 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ff794327fe1d4cbb9c769076ffe73dd5'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:51.134407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1115/2211] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:50 2022] DELETE /identity/v3/users/ff794327fe1d4cbb9c769076ffe73dd5 => generated 0 bytes in 968 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:52.049335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.051018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:52.065186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b957640b-faac-49e9-9c04-7f10314c42a7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.133662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:52.140101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.140959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.141305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.141516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] PATH_INFO: `/v3/projects/055d13f46574420283883011c8119db3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.142718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=055d13f46574420283883011c8119db3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:52.144682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:52.306240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '055d13f46574420283883011c8119db3', 'request_id': 'req-01cdc140-31f2-435e-a97d-adc4dd4713c2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:52.313404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '055d13f46574420283883011c8119db3', 'request_id': 'req-01cdc140-31f2-435e-a97d-adc4dd4713c2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:52.377177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '055d13f46574420283883011c8119db3', 'request_id': 'req-01cdc140-31f2-435e-a97d-adc4dd4713c2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:52.390644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] The token cache is being invalidate because project 055d13f46574420283883011c8119db3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:58:52.390934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:52.391648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-01cdc140-31f2-435e-a97d-adc4dd4713c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:52.393522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1097/2212] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:58:52 2022] DELETE /identity/v3/projects/055d13f46574420283883011c8119db3 => generated 0 bytes in 345 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:52.398417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.399048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.399259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.419951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:52.421373 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:52.482794 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1116/2213] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:52.491419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.491709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.492020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.511125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:52.512370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:52.520278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1098/2214] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:52.524331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.533856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314e5c98-3b5a-4cbb-b107-f0e0443dd933 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.538392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.539043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.539330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.539609 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.540371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:58:52.540686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:52.542343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:52.553937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1117/2215] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:58:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:58:52.558840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.568038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a297a5d4-be82-40ab-b9b0-96a351b018df None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.572753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.573973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.574256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.574533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.575233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:52.577176 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:52.691097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1099/2216] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:58:52 2022] POST /identity/v3/projects => generated 393 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:58:52.694563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.708522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2980b17c-98d4-4069-ac52-f1ef3d90575d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.734600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:52.785300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.785991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.786273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.786550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.787777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:52.789878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:52.895467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1118/2217] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:58:52 2022] POST /identity/v3/users => generated 297 bytes in 201 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:52.899474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.909878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f50cc78-ac61-494d-9350-d3e82a9dd6be admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.914473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.915159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.915444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.915721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.916516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:52.918694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:52.931212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1100/2218] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:52.935661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.945959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2577701-8d7d-4ed8-944e-a5ba174c2147 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.950939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa495a2f-decd-4604-b04f-b131805f07b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.951997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495a2f-decd-4604-b04f-b131805f07b0 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.951997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495a2f-decd-4604-b04f-b131805f07b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.952171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa495a2f-decd-4604-b04f-b131805f07b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:52.953007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa495a2f-decd-4604-b04f-b131805f07b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:52.954599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa495a2f-decd-4604-b04f-b131805f07b0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:52.965157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1119/2219] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:52 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:52.970492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:52.983351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9486569f-6687-4adf-8801-88e4ec2c4ee7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:52.988019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:52.988908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:52.989188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:52.989503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] PATH_INFO: `/v3/projects/f6dc70187df44694b91d30bc274698b2/users/1170eaaaed4d47f4abf20c95bdac58b2/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.008851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f6dc70187df44694b91d30bc274698b2, user_id=1170eaaaed4d47f4abf20c95bdac58b2, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.018467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.095659 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1101/2220] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:52 2022] PUT /identity/v3/projects/f6dc70187df44694b91d30bc274698b2/users/1170eaaaed4d47f4abf20c95bdac58b2/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:58:53.106447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.106658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.106962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.135819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] MFA Rules not processed for user `1170eaaaed4d47f4abf20c95bdac58b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:53.137288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:53.204572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1120/2221] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:58:53.208946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:53.278967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0021f79f-b9d8-4529-847e-0fd274c40140 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:53.283843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:53.284532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.284814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.285086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.285812 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.287723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.393719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1102/2222] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:58:53 2022] POST /identity/v3/projects => generated 393 bytes in 185 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:58:53.397967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:53.411618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7af59748-2430-4270-a5df-58d35cbde001 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:53.479346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:53.479977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.480271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.480553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.481273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.483603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.611417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1121/2223] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:58:53 2022] POST /identity/v3/users => generated 303 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:58:53.614273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:53.625633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed2e7856-061a-4a18-8aee-e73cdbfbac12 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:53.633264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:53.633985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.641603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.646776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.646776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.646776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.648838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:53.662859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1103/2224] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:53.667232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:53.681439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70e5eae7-5f14-4509-b4cd-d8d0d404a76f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:53.686397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:53.687295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.687584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.687899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] PATH_INFO: `/v3/projects/c1861ec1ba1f4097917051f19bcbaee4/users/2f8ea4329ba443e6a934c5291177bf28/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.704209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1861ec1ba1f4097917051f19bcbaee4, user_id=2f8ea4329ba443e6a934c5291177bf28, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.706950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.710289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:53.775361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1122/2225] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:53 2022] PUT /identity/v3/projects/c1861ec1ba1f4097917051f19bcbaee4/users/2f8ea4329ba443e6a934c5291177bf28/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:53.779431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:53.790683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc242adf-4049-474b-949b-0367b22e3e8e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:53.865779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c541c31-80b3-483c-aca3-02749f7b6e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:53.866476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c541c31-80b3-483c-aca3-02749f7b6e99 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.866764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c541c31-80b3-483c-aca3-02749f7b6e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.867042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c541c31-80b3-483c-aca3-02749f7b6e99 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.867786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c541c31-80b3-483c-aca3-02749f7b6e99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.869648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c541c31-80b3-483c-aca3-02749f7b6e99 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.881101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1104/2226] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:58:53 2022] GET /identity/v3/roles => generated 1420 bytes in 102 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:58:53.885161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:53.896355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e25c7-04a7-4656-a71a-16aff1bc30f0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:53.901491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:53.902385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.902722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.903008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] PATH_INFO: `/v3/projects/c1861ec1ba1f4097917051f19bcbaee4/users/2f8ea4329ba443e6a934c5291177bf28/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.905133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] RBAC: Authorizing `identity:create_grant(project_id=c1861ec1ba1f4097917051f19bcbaee4, user_id=2f8ea4329ba443e6a934c5291177bf28, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:53.910792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:53.920035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1123/2227] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:58:53 2022] PUT /identity/v3/projects/c1861ec1ba1f4097917051f19bcbaee4/users/2f8ea4329ba443e6a934c5291177bf28/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:53.934006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:53.934006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:53.934006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:53.969892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] MFA Rules not processed for user `2f8ea4329ba443e6a934c5291177bf28`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:53.971236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:54.042843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1105/2228] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:53 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:54.054124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:54.056397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:54.143416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f465d147-dc5e-41ef-a7a1-6cfc1c7fe00a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:54.147713 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:54.148442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:54.148760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:54.149237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:54.159251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:54.160786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:54.171773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1124/2229] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:54 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:54.218728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:54.219793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:54.233697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45df2e0a-ebec-4556-a89b-f6bfb4f96e09 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:54.239127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd22c66-d633-4050-a5e5-2c62f14ad6ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:54.239754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd22c66-d633-4050-a5e5-2c62f14ad6ed service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:54.240063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd22c66-d633-4050-a5e5-2c62f14ad6ed service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:54.240343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd22c66-d633-4050-a5e5-2c62f14ad6ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:54.299225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd22c66-d633-4050-a5e5-2c62f14ad6ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:54.300739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd22c66-d633-4050-a5e5-2c62f14ad6ed service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:54.311326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1106/2230] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:54 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:58:59.439964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:59.514885 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97929b02-81e7-43ca-b44f-b070f082c404 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:59.521927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:59.522972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:59.523381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:59.523665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] PATH_INFO: `/v3/users/1170eaaaed4d47f4abf20c95bdac58b2` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:59.525508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1170eaaaed4d47f4abf20c95bdac58b2)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:59.528735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:58:59.705744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1170eaaaed4d47f4abf20c95bdac58b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:59.706640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1170eaaaed4d47f4abf20c95bdac58b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:59.780087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1170eaaaed4d47f4abf20c95bdac58b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:59.790020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1170eaaaed4d47f4abf20c95bdac58b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:59.796344 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:INSERT=2,keystone:DELETE=8 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:59.804706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1170eaaaed4d47f4abf20c95bdac58b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:58:59.811290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1125/2231] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:58:59 2022] DELETE /identity/v3/users/1170eaaaed4d47f4abf20c95bdac58b2 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:58:59.815724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:59.816232 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:59.816334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:59.833814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:58:59.835031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:59.848001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:58:59.906099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1107/2232] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:58:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:58:59.915742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:58:59.919237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:58:59.982166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1f9b7b6-092b-4810-b7d3-15e339c07701 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:58:59.986016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:58:59.986717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:58:59.987103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:58:59.987328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:58:59.996735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:58:59.998169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:00.008107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1126/2233] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:58:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:59:00.485686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:00.487174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:00.506129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a60f832b-1cca-4ee1-94ef-990184a6f56c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:00.520774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:00.522028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:00.522756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:00.523193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] PATH_INFO: `/v3/projects/f6dc70187df44694b91d30bc274698b2` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:00.524615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] RBAC: Authorizing `identity:delete_project(project_id=f6dc70187df44694b91d30bc274698b2)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:00.527840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:00.642909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f6dc70187df44694b91d30bc274698b2', 'request_id': 'req-170ea811-d343-4cf8-8d57-9227e4965c6e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:00.648771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f6dc70187df44694b91d30bc274698b2', 'request_id': 'req-170ea811-d343-4cf8-8d57-9227e4965c6e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:00.737120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f6dc70187df44694b91d30bc274698b2', 'request_id': 'req-170ea811-d343-4cf8-8d57-9227e4965c6e'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:00.753162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] The token cache is being invalidate because project f6dc70187df44694b91d30bc274698b2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:59:00.753455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:00.754181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:00.755924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1108/2234] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:59:00 2022] DELETE /identity/v3/projects/f6dc70187df44694b91d30bc274698b2 => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:00.760050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:00.770810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661ef644-b28d-4dbb-a3ea-607f55597023 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:00.835649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:00.836575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:00.836865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:00.837480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] PATH_INFO: `/v3/users/2f8ea4329ba443e6a934c5291177bf28` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:00.838762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2f8ea4329ba443e6a934c5291177bf28)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:00.840802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:00.998665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2f8ea4329ba443e6a934c5291177bf28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:00.999487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2f8ea4329ba443e6a934c5291177bf28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:01.057531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2f8ea4329ba443e6a934c5291177bf28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:01.065469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2f8ea4329ba443e6a934c5291177bf28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:01.078640 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-13639c84-3599-4d96-b934-b550108f6614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2f8ea4329ba443e6a934c5291177bf28'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:01.085022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1127/2235] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:59:00 2022] DELETE /identity/v3/users/2f8ea4329ba443e6a934c5291177bf28 => generated 0 bytes in 325 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:01.878910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:01.881015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:01.895126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-170ea811-d343-4cf8-8d57-9227e4965c6e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:01.959202 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:01.960139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:01.960426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:01.960703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] PATH_INFO: `/v3/projects/c1861ec1ba1f4097917051f19bcbaee4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:01.961886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c1861ec1ba1f4097917051f19bcbaee4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:01.964626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.062289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:02.065906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c1861ec1ba1f4097917051f19bcbaee4', 'request_id': 'req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:02.071556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c1861ec1ba1f4097917051f19bcbaee4', 'request_id': 'req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:02.106681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212f535-7700-4584-88a8-6cf7f318e676 None None] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.107213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212f535-7700-4584-88a8-6cf7f318e676 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.107483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212f535-7700-4584-88a8-6cf7f318e676 None None] PATH_INFO: `/` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.109160 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1128/2236] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jun 16 11:59:02.117602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.117947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.118274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.130681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:02.155992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c1861ec1ba1f4097917051f19bcbaee4', 'request_id': 'req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:02.169151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] The token cache is being invalidate because project c1861ec1ba1f4097917051f19bcbaee4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:59:02.169564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:02.169925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:02.170379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-540ac9c4-8ef4-4e2a-a16a-a0950d74e6c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:02.171136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.174359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1109/2237] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:59:01 2022] DELETE /identity/v3/projects/c1861ec1ba1f4097917051f19bcbaee4 => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:02.177768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36606b83-64da-44c9-9edd-ccb1bf8dabd6 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.178062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36606b83-64da-44c9-9edd-ccb1bf8dabd6 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.178346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36606b83-64da-44c9-9edd-ccb1bf8dabd6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.199497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-36606b83-64da-44c9-9edd-ccb1bf8dabd6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:02.205321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-36606b83-64da-44c9-9edd-ccb1bf8dabd6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.257512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1129/2238] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 141 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:02.268850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1110/2239] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:02.269327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.270925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.276473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.276864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.277362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.296023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:02.297439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.305708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1111/2240] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:02.310501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.321479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a647f603-9dfe-4c18-804b-8f716fef9591 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.328154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.328842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.329559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.329852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.330587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:59:02.330913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.332567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.336001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96853511-cff7-4fdf-a5d1-9703dc05b450 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.339811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.340520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.340808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.341089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.345924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1112/2241] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:59:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 11:59:02.350560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.352126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.353590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.359980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f1e859a-0d89-43d5-b6f9-8d25bb36ee19 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.364560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550cf5f0-3564-4ee1-9cee-eb927d6adcde admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.367057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cf5f0-3564-4ee1-9cee-eb927d6adcde admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.367057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cf5f0-3564-4ee1-9cee-eb927d6adcde admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.367057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550cf5f0-3564-4ee1-9cee-eb927d6adcde admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.367337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1130/2242] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:59:02.367446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cf5f0-3564-4ee1-9cee-eb927d6adcde admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.368628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550cf5f0-3564-4ee1-9cee-eb927d6adcde admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.448749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1113/2243] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/projects => generated 359 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:59:02.452940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.463112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64a6e422-7db9-4f15-8570-77a5b519e15c service glance] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.519838 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.520485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.520767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.521042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.521797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.523706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.581182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02a103-7b3f-45f8-ac95-6849a39b6bc2 None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.581713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02a103-7b3f-45f8-ac95-6849a39b6bc2 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.582071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f02a103-7b3f-45f8-ac95-6849a39b6bc2 None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.583781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1114/2244] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:59:02.588695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.589334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.590018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.597588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1131/2245] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/users => generated 280 bytes in 145 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:59:02.601496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.613465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] MFA Rules not processed for user `516b221cb1794430befadcde5f9fd77b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:02.613654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b3b08d9-3ce2-4bc9-b25d-2fb0eb05a4c2 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.614706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.621966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.626617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.626617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.626617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.626617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.626617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.639816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1132/2246] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:59:02.640727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.651944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4e2537b-2c8c-474a-a359-b7351cc2280c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.657212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.658018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.658406 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.658807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.659689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.662126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.672871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1115/2247] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:02.677334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1133/2248] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:59:02.681251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.687575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:02.689451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.692774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60a8f004-2395-4810-8133-e35fd226f5ad None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.698089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d8d61f-857b-4586-b1e4-41479847c255 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.699073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8d61f-857b-4586-b1e4-41479847c255 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.699422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8d61f-857b-4586-b1e4-41479847c255 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.699762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d8d61f-857b-4586-b1e4-41479847c255 admin admin] PATH_INFO: `/v3/projects/e1fe5985e9b446e8a7e9e7cd15160681/users/bc7b8470f2a84d55a587a26cfe4d9eaa/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.705087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-639ea3ec-2058-49c1-a7d5-95c297499bf9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:02.708901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4096e4a6-9133-421f-8476-5c15e1d27cad service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:02.709555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096e4a6-9133-421f-8476-5c15e1d27cad service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.709846 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096e4a6-9133-421f-8476-5c15e1d27cad service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.710131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4096e4a6-9133-421f-8476-5c15e1d27cad service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.717571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8d61f-857b-4586-b1e4-41479847c255 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e1fe5985e9b446e8a7e9e7cd15160681, user_id=bc7b8470f2a84d55a587a26cfe4d9eaa, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.718266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096e4a6-9133-421f-8476-5c15e1d27cad service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:02.720779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d8d61f-857b-4586-b1e4-41479847c255 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.720972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4096e4a6-9133-421f-8476-5c15e1d27cad service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:02.729370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1134/2249] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:02 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:59:02.809681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1116/2250] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:59:02 2022] PUT /identity/v3/projects/e1fe5985e9b446e8a7e9e7cd15160681/users/bc7b8470f2a84d55a587a26cfe4d9eaa/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 129 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 11:59:02.820622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.820910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.821192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.850562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56c71a-ee2a-4220-9d62-eebedac32b7d None None] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.850861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56c71a-ee2a-4220-9d62-eebedac32b7d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.851126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e56c71a-ee2a-4220-9d62-eebedac32b7d None None] PATH_INFO: `/` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.851229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] MFA Rules not processed for user `bc7b8470f2a84d55a587a26cfe4d9eaa`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:02.852417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.852606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1117/2251] 10.222.0.40 () {54 vars in 921 bytes} [Thu Jun 16 11:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jun 16 11:59:02.868913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:02.869201 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:02.869996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:02.891349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] MFA Rules not processed for user `54986964e91b4af7997e09d4682cc31e`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:02.893884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:02.905210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:02.916221 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:02.962028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1135/2252] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3226 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:03.004945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:03.006445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:03.012013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1118/2253] 10.222.0.40 () {58 vars in 1018 bytes} [Thu Jun 16 11:59:02 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 144 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:03.025509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:03.027475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:03.090892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b5802-1b08-47a9-9886-6d569ce2e354 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:03.095802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:03.096374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49d9c691-25e4-4e11-b629-073109a1a9d5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:03.096492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:03.096784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:03.097059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:03.101311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42cd7c88-79e7-4fbc-989b-6591ef674636 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:03.102228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7c88-79e7-4fbc-989b-6591ef674636 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:03.102755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7c88-79e7-4fbc-989b-6591ef674636 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:03.103127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42cd7c88-79e7-4fbc-989b-6591ef674636 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:03.106290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:03.107834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:03.114594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7c88-79e7-4fbc-989b-6591ef674636 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:03.116430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42cd7c88-79e7-4fbc-989b-6591ef674636 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:03.117857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1136/2254] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:03 2022] GET /identity/v3/auth/tokens => generated 3226 bytes in 114 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:59:03.130165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1119/2255] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:03 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:59:03.137832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:03.199548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:03.237357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcd7acec-9c3d-48ad-9336-96805ba052ab service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:03.247081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:03.248020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:03.248318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:03.248730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] PATH_INFO: `/v3/users/bc7b8470f2a84d55a587a26cfe4d9eaa` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:03.251887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] RBAC: Authorizing `identity:delete_user(user_id=bc7b8470f2a84d55a587a26cfe4d9eaa)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:03.253318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:03.470363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'bc7b8470f2a84d55a587a26cfe4d9eaa'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:03.471016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'bc7b8470f2a84d55a587a26cfe4d9eaa'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:03.550214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'bc7b8470f2a84d55a587a26cfe4d9eaa'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:03.558189 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'bc7b8470f2a84d55a587a26cfe4d9eaa'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:03.570530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'bc7b8470f2a84d55a587a26cfe4d9eaa'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:03.576994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1137/2256] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:59:03 2022] DELETE /identity/v3/users/bc7b8470f2a84d55a587a26cfe4d9eaa => generated 0 bytes in 440 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:03.581786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:03.582075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:03.582353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:03.600452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:03.601595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:03.662533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1120/2257] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:03.673073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:03.674481 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:03.731960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbd17e12-eec2-4540-8f15-f7f769bd5130 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:03.735529 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66bc3716-d8fb-424e-8b61-78edc14151ea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:03.736208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc3716-d8fb-424e-8b61-78edc14151ea service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:03.736498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc3716-d8fb-424e-8b61-78edc14151ea service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:03.736774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66bc3716-d8fb-424e-8b61-78edc14151ea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:03.745730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc3716-d8fb-424e-8b61-78edc14151ea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:03.747156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66bc3716-d8fb-424e-8b61-78edc14151ea service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:03.757078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1138/2258] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:04.202989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:04.204541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:04.217940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb00e69-ccbe-4de8-b425-c2cd2bf993ac None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:04.224887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:04.225804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:04.226131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:04.226358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] PATH_INFO: `/v3/projects/e1fe5985e9b446e8a7e9e7cd15160681` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:04.228169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] RBAC: Authorizing `identity:delete_project(project_id=e1fe5985e9b446e8a7e9e7cd15160681)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:04.230826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:04.353123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:04.356328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e1fe5985e9b446e8a7e9e7cd15160681', 'request_id': 'req-c253e8f5-6961-4aa8-9f18-d34da70403bd'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:04.363233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e1fe5985e9b446e8a7e9e7cd15160681', 'request_id': 'req-c253e8f5-6961-4aa8-9f18-d34da70403bd'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:04.448107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e1fe5985e9b446e8a7e9e7cd15160681', 'request_id': 'req-c253e8f5-6961-4aa8-9f18-d34da70403bd'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:04.466270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] The token cache is being invalidate because project e1fe5985e9b446e8a7e9e7cd15160681 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:59:04.466903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:04.468690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c253e8f5-6961-4aa8-9f18-d34da70403bd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:04.476644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1121/2259] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:59:04 2022] DELETE /identity/v3/projects/e1fe5985e9b446e8a7e9e7cd15160681 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:04.477161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:04.477645 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:04.477741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:04.498970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:04.500274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:04.572764 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1139/2260] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:04.577582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:04.577582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:04.577582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:04.597854 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:04.599730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:04.611573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1122/2261] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:04.616787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:04.636576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1a91e6a-da7e-47f5-81cb-32dabeb82336 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:04.644587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:04.645741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:04.646251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:04.646763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:04.648139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:59:04.648709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:04.652397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:04.664698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:04.674434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1140/2262] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:59:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 59 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:59:04.684222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:04.706480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-943853dd-a412-47e0-a700-17d2ef625fad None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:04.712470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:04.712690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:04.713012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:04.714668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:04.715471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:04.717624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:04.819637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1123/2263] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:59:04 2022] POST /identity/v3/projects => generated 385 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:59:04.824682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:04.837707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569def52-35c5-4f61-b02f-c29cb8be31e4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:04.908686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:04.909472 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:04.909827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:04.910173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:04.910966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:04.913142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:05.013789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1141/2264] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:59:04 2022] POST /identity/v3/users => generated 293 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:59:05.017862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:05.028525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bed4b9f-f4bd-4288-8fff-4f6f0d3489d7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:05.037715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:05.038414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:05.038703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:05.039010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:05.039742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:05.041441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:05.057682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1124/2265] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 11:59:05.061399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:05.076978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a5e5b69-64f5-420f-a3ff-2164d27e10fb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:05.084570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c64594f-cc5f-4bf2-992a-564dcafa7c11 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:05.085255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64594f-cc5f-4bf2-992a-564dcafa7c11 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:05.093204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64594f-cc5f-4bf2-992a-564dcafa7c11 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:05.093816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c64594f-cc5f-4bf2-992a-564dcafa7c11 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:05.094605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64594f-cc5f-4bf2-992a-564dcafa7c11 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:05.096285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c64594f-cc5f-4bf2-992a-564dcafa7c11 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:05.116827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1142/2266] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:05 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:59:05.118080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:05.129385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae1d4282-a9f5-4f94-9af1-6f957748c8db admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:05.136084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:05.136991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:05.137274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:05.137894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] PATH_INFO: `/v3/projects/ecb4c8fc3a524fc18ebee7d56d8d7278/users/96541eb431514859aef2bbfc99a1eba0/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:05.157524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ecb4c8fc3a524fc18ebee7d56d8d7278, user_id=96541eb431514859aef2bbfc99a1eba0, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:05.160198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:05.224401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1125/2267] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:59:05 2022] PUT /identity/v3/projects/ecb4c8fc3a524fc18ebee7d56d8d7278/users/96541eb431514859aef2bbfc99a1eba0/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:05.234835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:05.235139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:05.235466 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:05.266509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] MFA Rules not processed for user `96541eb431514859aef2bbfc99a1eba0`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:05.267849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:05.343946 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1143/2268] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:05 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:05.356273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:05.358006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:05.418336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb53ce21-7b59-4ad0-977e-63f4e60be9eb admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:05.422130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5bd0a080-f3e4-4199-a438-5566ebb15a15 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:05.422790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0a080-f3e4-4199-a438-5566ebb15a15 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:05.423076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0a080-f3e4-4199-a438-5566ebb15a15 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:05.423360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bd0a080-f3e4-4199-a438-5566ebb15a15 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:05.433125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0a080-f3e4-4199-a438-5566ebb15a15 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:05.434566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5bd0a080-f3e4-4199-a438-5566ebb15a15 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:05.443478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1126/2269] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 11:59:06.614186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:06.615655 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:06.688077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:06.714931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-437a1f85-451f-4847-9e0c-87e1754bd346 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:06.720445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:06.721141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:06.721566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:06.721855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:06.723709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:06.735561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:06.737290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:06.747974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1144/2270] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:06 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:07.453996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:07.453996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:07.453996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:07.518880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:07.520255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:07.577993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=4,keystone:SELECT=93,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:07.615572 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1127/2271] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:59:07 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:07.628284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:07.631063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:07.746310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21f3ab55-dbe6-46dd-926f-9e35884d931a service placement] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:07.751791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b4c8e-78bd-4ae6-9157-bbdc7960f402 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:07.752442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4c8e-78bd-4ae6-9157-bbdc7960f402 service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:07.752950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4c8e-78bd-4ae6-9157-bbdc7960f402 service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:07.753067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b4c8e-78bd-4ae6-9157-bbdc7960f402 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:07.762265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4c8e-78bd-4ae6-9157-bbdc7960f402 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:07.763978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e7b4c8e-78bd-4ae6-9157-bbdc7960f402 service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:07.772690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1145/2272] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:07 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 11:59:08.455705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:08.529501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f4cd5a-2c8d-45ce-8335-61002d87c588 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:08.534440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:08.535334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:08.535619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:08.535925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] PATH_INFO: `/v3/users/96541eb431514859aef2bbfc99a1eba0` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:08.537141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] RBAC: Authorizing `identity:delete_user(user_id=96541eb431514859aef2bbfc99a1eba0)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:08.539068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:08.741047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96541eb431514859aef2bbfc99a1eba0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:08.741767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96541eb431514859aef2bbfc99a1eba0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:08.813344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96541eb431514859aef2bbfc99a1eba0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:08.821279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96541eb431514859aef2bbfc99a1eba0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:08.837012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96541eb431514859aef2bbfc99a1eba0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:08.844670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1128/2273] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:59:08 2022] DELETE /identity/v3/users/96541eb431514859aef2bbfc99a1eba0 => generated 0 bytes in 389 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:08.849254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:08.849739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:08.849858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:08.870185 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:08.871661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:08.938238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1146/2274] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:08.948502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:08.949930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:09.023782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e26c3bbd-44f0-474b-83cc-43617acc5e4e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:09.028008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af84a64-cb35-4205-bd9d-cd3326b8efe9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:09.028668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84a64-cb35-4205-bd9d-cd3326b8efe9 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:09.028953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84a64-cb35-4205-bd9d-cd3326b8efe9 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:09.029230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af84a64-cb35-4205-bd9d-cd3326b8efe9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:09.043606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af84a64-cb35-4205-bd9d-cd3326b8efe9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:09.045081 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af84a64-cb35-4205-bd9d-cd3326b8efe9 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:09.055332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1129/2275] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:09.520346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:09.523128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:09.547938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09873c2b-2827-4f1d-b9eb-03f325f12b97 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:09.554716 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:09.555647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:09.556220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:09.556553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] PATH_INFO: `/v3/projects/ecb4c8fc3a524fc18ebee7d56d8d7278` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:09.557765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] RBAC: Authorizing `identity:delete_project(project_id=ecb4c8fc3a524fc18ebee7d56d8d7278)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:09.559821 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:09.670957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:09.693522 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ecb4c8fc3a524fc18ebee7d56d8d7278', 'request_id': 'req-7b82be74-fcbf-4823-b84d-e14040d513ce'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:09.694321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ecb4c8fc3a524fc18ebee7d56d8d7278', 'request_id': 'req-7b82be74-fcbf-4823-b84d-e14040d513ce'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:09.700710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ecb4c8fc3a524fc18ebee7d56d8d7278', 'request_id': 'req-7b82be74-fcbf-4823-b84d-e14040d513ce'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:09.871748 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] The token cache is being invalidate because project ecb4c8fc3a524fc18ebee7d56d8d7278 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:59:09.875895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:09.875895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7b82be74-fcbf-4823-b84d-e14040d513ce admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:09.875895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1147/2276] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:59:09 2022] DELETE /identity/v3/projects/ecb4c8fc3a524fc18ebee7d56d8d7278 => generated 0 bytes in 355 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:09.879386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:09.879841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:09.879982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:09.892799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:09.898671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:09.899890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:09.975228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1130/2277] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:09.983291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:09.983644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:09.983945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.007644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:10.009275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:10.019249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1148/2278] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:10.028589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.041410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89d2a48a-9b95-40b8-a62a-47cbd35d0734 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.046969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.047911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.048452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.048892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.049809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:59:10.050117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.051802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.068136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1131/2279] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:59:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:59:10.073399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.083230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998ad4b-0e1f-44a9-b6df-e4018b76197b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.089013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.089698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.090035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.090386 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.091163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.093196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.181548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1149/2280] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:59:10 2022] POST /identity/v3/projects => generated 399 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 11:59:10.186420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.202337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-512b0879-472c-4c65-8efb-d0e8f7e6c32a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.267143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.268082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.268464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.268790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.269545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.271774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.377516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1132/2281] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:59:10 2022] POST /identity/v3/users => generated 300 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:59:10.381395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.391904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19b935e2-bc11-4ae2-be4b-8d58e9d36c45 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.396612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.397298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.397690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.398059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.398800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.400933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.411599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1150/2282] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:59:10.417303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.433139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb91da42-c250-425d-b730-d48548d0f05e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.438665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bfdef30-e6a7-4dfc-88ec-bd8898d0c6be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.439560 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdef30-e6a7-4dfc-88ec-bd8898d0c6be admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.440023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdef30-e6a7-4dfc-88ec-bd8898d0c6be admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.440464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bfdef30-e6a7-4dfc-88ec-bd8898d0c6be admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.441370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfdef30-e6a7-4dfc-88ec-bd8898d0c6be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.443708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bfdef30-e6a7-4dfc-88ec-bd8898d0c6be admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.455300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1133/2283] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:59:10.459493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.471238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b34115c-ae16-4405-995d-28775e336ab6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.476304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.477428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.477783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.478065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] PATH_INFO: `/v3/projects/1d772db4f8af4123a033674e0d26edc6/users/48e3a675ea2f4ad4a3d9215521b86c2b/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.493919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] RBAC: Authorizing `identity:create_grant(project_id=1d772db4f8af4123a033674e0d26edc6, user_id=48e3a675ea2f4ad4a3d9215521b86c2b, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.496661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.578949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1151/2284] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:59:10 2022] PUT /identity/v3/projects/1d772db4f8af4123a033674e0d26edc6/users/48e3a675ea2f4ad4a3d9215521b86c2b/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:10.590208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.590510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.590849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.620283 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] MFA Rules not processed for user `48e3a675ea2f4ad4a3d9215521b86c2b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:10.621521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:10.695940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1134/2285] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:10 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:10.715637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:10.717403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:10.786301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9852aa00-552a-46cc-bd6a-4577fe4a43db admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:10.790165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-075109f2-6e3c-4fe4-a460-a42096c6181d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:10.791418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075109f2-6e3c-4fe4-a460-a42096c6181d service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:10.791770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075109f2-6e3c-4fe4-a460-a42096c6181d service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:10.792407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-075109f2-6e3c-4fe4-a460-a42096c6181d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:10.805241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075109f2-6e3c-4fe4-a460-a42096c6181d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:10.806868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-075109f2-6e3c-4fe4-a460-a42096c6181d service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:10.817783 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1152/2286] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:10 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:20.507256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:20.512003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:20.571045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba94d36c-2992-44c4-93de-6db5bc190a91 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:20.576373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:20.577263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:20.577547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:20.577825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] PATH_INFO: `/v3/users/48e3a675ea2f4ad4a3d9215521b86c2b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:20.579042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] RBAC: Authorizing `identity:delete_user(user_id=48e3a675ea2f4ad4a3d9215521b86c2b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:20.581020 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:20.806824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48e3a675ea2f4ad4a3d9215521b86c2b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:20.807491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48e3a675ea2f4ad4a3d9215521b86c2b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:20.812869 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=62,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:20.858204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48e3a675ea2f4ad4a3d9215521b86c2b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:20.866342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48e3a675ea2f4ad4a3d9215521b86c2b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:20.880315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48e3a675ea2f4ad4a3d9215521b86c2b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:20.887329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1135/2287] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:59:20 2022] DELETE /identity/v3/users/48e3a675ea2f4ad4a3d9215521b86c2b => generated 0 bytes in 380 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:20.891784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:20.899249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:20.899249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:20.911247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:20.913490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:20.972434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1153/2288] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:20.982084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:20.983548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:21.040124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a222242-153d-47f9-98a4-76f241841bac admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:21.043625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42b1e05d-cde3-46bc-89c9-23287c7dc3ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:21.044315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1e05d-cde3-46bc-89c9-23287c7dc3ff service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:21.044596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1e05d-cde3-46bc-89c9-23287c7dc3ff service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:21.044870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42b1e05d-cde3-46bc-89c9-23287c7dc3ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:21.053700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1e05d-cde3-46bc-89c9-23287c7dc3ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:21.055125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42b1e05d-cde3-46bc-89c9-23287c7dc3ff service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:21.064924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1136/2289] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:21.465781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:21.467202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:21.480694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94b3bde-14ad-4b02-8782-62153680cdb9 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:21.485454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:21.487355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:21.487355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:21.487355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] PATH_INFO: `/v3/projects/1d772db4f8af4123a033674e0d26edc6` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:21.488124 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1d772db4f8af4123a033674e0d26edc6)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:21.490170 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:21.571803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1d772db4f8af4123a033674e0d26edc6', 'request_id': 'req-2729df4e-0a00-4d83-a810-196e1e17bce0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:21.577769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1d772db4f8af4123a033674e0d26edc6', 'request_id': 'req-2729df4e-0a00-4d83-a810-196e1e17bce0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:21.583195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1d772db4f8af4123a033674e0d26edc6', 'request_id': 'req-2729df4e-0a00-4d83-a810-196e1e17bce0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:21.634443 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] The token cache is being invalidate because project 1d772db4f8af4123a033674e0d26edc6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:59:21.634804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:21.635467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:21.637143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1154/2290] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:59:21 2022] DELETE /identity/v3/projects/1d772db4f8af4123a033674e0d26edc6 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:25.611706 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279a03f-8340-4511-ba35-9bf797065643 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:25.612208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279a03f-8340-4511-ba35-9bf797065643 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:25.612312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1279a03f-8340-4511-ba35-9bf797065643 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:25.630220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1279a03f-8340-4511-ba35-9bf797065643 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:25.631431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1279a03f-8340-4511-ba35-9bf797065643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:25.653826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:25.695382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1137/2291] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:59:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:25.708563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:25.710012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:25.781710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2729df4e-0a00-4d83-a810-196e1e17bce0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:25.789662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:25.790435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:25.790788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:25.791154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:25.799544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:25.801171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:25.809139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1155/2292] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:59:26.566805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129323a-dade-4a9e-b4ca-5ee99a516fb5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:26.567099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129323a-dade-4a9e-b4ca-5ee99a516fb5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:26.567377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8129323a-dade-4a9e-b4ca-5ee99a516fb5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:26.585014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8129323a-dade-4a9e-b4ca-5ee99a516fb5 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:26.586207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8129323a-dade-4a9e-b4ca-5ee99a516fb5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:26.593595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1138/2293] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 11:59:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:26.607983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:26.617351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac94d90-e85b-4943-b97a-9e73efbefcbf service swift] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:26.623069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:26.623701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:26.624028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:26.624310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:26.632444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:26.634014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:26.638533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:26.642582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1156/2294] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 36 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 11:59:27.237357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fd037-854d-4de6-ad7f-4a20d5661019 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:27.237818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fd037-854d-4de6-ad7f-4a20d5661019 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:27.237939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f21fd037-854d-4de6-ad7f-4a20d5661019 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:27.255449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f21fd037-854d-4de6-ad7f-4a20d5661019 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:27.256777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f21fd037-854d-4de6-ad7f-4a20d5661019 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:27.264561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1139/2295] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:27.276972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:27.287969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8df0b3c0-e129-4e6c-8d09-3cab6e25cd11 service swift] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:27.293089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:27.293730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:27.294014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:27.294288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:27.302229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:27.303793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:27.311688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1157/2296] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 11:59:27.880036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396aaec-d117-4f9e-8e81-df6164d83ead None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:27.880571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396aaec-d117-4f9e-8e81-df6164d83ead None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:27.881016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d396aaec-d117-4f9e-8e81-df6164d83ead None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:27.906636 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d396aaec-d117-4f9e-8e81-df6164d83ead None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:27.907786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d396aaec-d117-4f9e-8e81-df6164d83ead None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:27.915521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1140/2297] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 11:59:27 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:27.927979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:27.939093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0769d35-bb46-48ec-b45c-d9fc9b15b0eb service swift] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:27.944068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:27.944701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:27.944985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:27.945260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:27.953350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:27.954944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:27.964181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1158/2298] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:27 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 11:59:30.284833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a02b-f940-4e81-80f0-5ab265185359 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:30.285344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a02b-f940-4e81-80f0-5ab265185359 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:30.285538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c43a02b-f940-4e81-80f0-5ab265185359 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:30.302559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7c43a02b-f940-4e81-80f0-5ab265185359 None None] MFA Rules not processed for user `9082354c3ab249f885e129b86799e508`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:30.303759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7c43a02b-f940-4e81-80f0-5ab265185359 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:30.357619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1141/2299] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:30 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:30.589382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:30.592080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:30.682361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8afabcc-7e88-497c-8b84-ee715a451684 service swift] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:30.695420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:30.697274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:30.697662 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:30.698069 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] PATH_INFO: `/v3/users/e32560e5ffa640d0b505834f090bd330` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:30.699492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] RBAC: Authorizing `identity:delete_user(user_id=e32560e5ffa640d0b505834f090bd330)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:30.701587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:30.900068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e32560e5ffa640d0b505834f090bd330'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:30.900830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e32560e5ffa640d0b505834f090bd330'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:30.972469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e32560e5ffa640d0b505834f090bd330'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:30.980710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e32560e5ffa640d0b505834f090bd330'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:30.994049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e32560e5ffa640d0b505834f090bd330'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:31.000252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1159/2300] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 11:59:30 2022] DELETE /identity/v3/users/e32560e5ffa640d0b505834f090bd330 => generated 0 bytes in 411 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:31.005069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.005364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.005668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.024136 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:31.025493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:31.085262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1142/2301] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:31.095019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:31.096453 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:31.153981 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8056281-8112-4ab6-911d-2d2e5ffbc09b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:31.157813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5ca480c-a79c-42e0-bb48-956f6d914f90 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:31.163122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca480c-a79c-42e0-bb48-956f6d914f90 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.163122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca480c-a79c-42e0-bb48-956f6d914f90 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.163122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5ca480c-a79c-42e0-bb48-956f6d914f90 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.168496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:31.171563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca480c-a79c-42e0-bb48-956f6d914f90 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:31.173536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5ca480c-a79c-42e0-bb48-956f6d914f90 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:31.192720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1160/2302] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:31.488129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:31.489591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:31.502292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b66d509f-667b-4453-887a-0af990c607f1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:31.507091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:31.508016 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.508303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.515449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] PATH_INFO: `/v3/projects/c6602646a8734415a2cd5c138ec99b29` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.515449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c6602646a8734415a2cd5c138ec99b29)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:31.515449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:31.633463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c6602646a8734415a2cd5c138ec99b29', 'request_id': 'req-29e20276-dec1-42bc-965e-ff8c88053a48'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:31.635122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:31.640357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c6602646a8734415a2cd5c138ec99b29', 'request_id': 'req-29e20276-dec1-42bc-965e-ff8c88053a48'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:31.724904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c6602646a8734415a2cd5c138ec99b29', 'request_id': 'req-29e20276-dec1-42bc-965e-ff8c88053a48'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:31.736516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] The token cache is being invalidate because project c6602646a8734415a2cd5c138ec99b29 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 11:59:31.736812 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:31.737453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-29e20276-dec1-42bc-965e-ff8c88053a48 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 11:59:31.739018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1143/2303] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 11:59:31 2022] DELETE /identity/v3/projects/c6602646a8734415a2cd5c138ec99b29 => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:31.743799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.744283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.744401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.763758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:31.765005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:31.827736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1161/2304] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:31.836045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.836401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.836718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.857152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:31.858381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:31.866553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1144/2305] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:31.870732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:31.880579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ee302e-b07b-4b71-90c0-72d42ae31efb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:31.886047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:31.886743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.887032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.887313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.888082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 11:59:31.888394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:31.889998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:31.902140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1162/2306] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 11:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 11:59:31.906431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:31.915847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca5d074a-9442-4f82-ae49-73da2548ae0c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:31.920589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:31.921239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:31.921525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:31.921811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:31.922501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:31.924447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:31.982595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1145/2307] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 11:59:31 2022] POST /identity/v3/projects => generated 381 bytes in 74 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:59:31.984271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:31.994797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af06b296-500e-4858-8ee8-e1c466fa7fb9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:32.051229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:32.051933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:32.052225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:32.052506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:32.053199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:32.055166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:32.159228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1163/2308] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 11:59:31 2022] POST /identity/v3/users => generated 291 bytes in 176 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 11:59:32.163231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:32.173773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ccfd58-ef4a-48cd-8656-43580650e57c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:32.178584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:32.179287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:32.179578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:32.179848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:32.181241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:32.182944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:32.193780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1146/2309] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:59:32.198454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:32.208547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4960bc11-c646-4061-994f-ab1b2b002227 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:32.213195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2277f12-db6f-4b75-a695-b1d1d8c516f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:32.213847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277f12-db6f-4b75-a695-b1d1d8c516f6 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:32.214132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277f12-db6f-4b75-a695-b1d1d8c516f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:32.215325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2277f12-db6f-4b75-a695-b1d1d8c516f6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:32.215325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2277f12-db6f-4b75-a695-b1d1d8c516f6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:32.216737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2277f12-db6f-4b75-a695-b1d1d8c516f6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:32.226667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1164/2310] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 11:59:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 11:59:32.230905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:32.241240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cc96cc-cb51-4692-8cf1-5c3f9f8e9a2a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:32.246472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:32.247357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:32.247662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:32.247976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] PATH_INFO: `/v3/projects/82c80e6e109741d1bd83c7b81368a87f/users/5e69fc12fcc34fa39f67dd156cd860a4/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:32.266358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] RBAC: Authorizing `identity:create_grant(project_id=82c80e6e109741d1bd83c7b81368a87f, user_id=5e69fc12fcc34fa39f67dd156cd860a4, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:32.269356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:32.344637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1147/2311] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 11:59:32 2022] PUT /identity/v3/projects/82c80e6e109741d1bd83c7b81368a87f/users/5e69fc12fcc34fa39f67dd156cd860a4/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 11:59:32.350541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29cc7d-1fc7-4d12-8f80-e66228b09097 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:32.350837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29cc7d-1fc7-4d12-8f80-e66228b09097 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:32.351122 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa29cc7d-1fc7-4d12-8f80-e66228b09097 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:32.369621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-aa29cc7d-1fc7-4d12-8f80-e66228b09097 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:32.370861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-aa29cc7d-1fc7-4d12-8f80-e66228b09097 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:32.429760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1165/2312] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:32.439986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:32.441720 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:32.501155 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aee5db50-2a98-44f8-814e-c72409878267 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:32.504908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:32.505556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:32.505860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:32.506137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:32.515356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:32.516863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:32.527327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1148/2313] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:33.987511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c106c-9211-4085-a07b-18dc407321db None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:33.988003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c106c-9211-4085-a07b-18dc407321db None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:33.988261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-701c106c-9211-4085-a07b-18dc407321db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:33.994823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:34.023232 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-701c106c-9211-4085-a07b-18dc407321db None None] MFA Rules not processed for user `5e69fc12fcc34fa39f67dd156cd860a4`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:34.024960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-701c106c-9211-4085-a07b-18dc407321db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:34.099087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1166/2314] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:33 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 11:59:34.110585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:34.112141 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:34.182699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee6db2cb-6f3a-4866-90db-83f1414cbfb5 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:34.186959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:34.188336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:34.188681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:34.188966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:34.206181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:34.207829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:34.220353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1149/2315] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 11:59:34 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 110 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 11:59:35.523491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9458f39-9d0f-4ceb-a3d2-c495c203b16e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:35.524165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9458f39-9d0f-4ceb-a3d2-c495c203b16e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:35.524165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9458f39-9d0f-4ceb-a3d2-c495c203b16e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:35.542592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e9458f39-9d0f-4ceb-a3d2-c495c203b16e None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:35.543921 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e9458f39-9d0f-4ceb-a3d2-c495c203b16e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:35.601950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1167/2316] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:35.617054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:35.617054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:35.639491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=3,keystone:SELECT=94,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:35.688868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-770a1c80-490a-4823-85a2-f9709a02694d service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:35.695360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cb91c-5e35-487f-872a-13a42362405d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:35.696156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb91c-5e35-487f-872a-13a42362405d service swift] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:35.696450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb91c-5e35-487f-872a-13a42362405d service swift] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:35.696723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cb91c-5e35-487f-872a-13a42362405d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:35.705490 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb91c-5e35-487f-872a-13a42362405d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:35.707076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cb91c-5e35-487f-872a-13a42362405d service swift] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:35.715442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1150/2317] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:35 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 11:59:35.856218 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:35.856469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:35.856761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:35.879737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] MFA Rules not processed for user `5e69fc12fcc34fa39f67dd156cd860a4`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:35.881117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:35.888268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1168/2318] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 11:59:35 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:45.760472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=23 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:45.869996 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=59 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 11:59:58.258115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b3da-378a-418a-accf-5072a5605956 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:58.258634 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b3da-378a-418a-accf-5072a5605956 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:58.258739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35a9b3da-378a-418a-accf-5072a5605956 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:58.276655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-35a9b3da-378a-418a-accf-5072a5605956 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 11:59:58.277861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-35a9b3da-378a-418a-accf-5072a5605956 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 11:59:58.284987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1151/2319] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 11:59:58 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 11:59:58.308308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 11:59:58.320692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7640a02b-378b-4693-8923-fd6beb583a31 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 11:59:58.325765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d670581-ccae-4696-a4fa-cfbf3650826f service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 11:59:58.326413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d670581-ccae-4696-a4fa-cfbf3650826f service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 11:59:58.326697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d670581-ccae-4696-a4fa-cfbf3650826f service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 11:59:58.326969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d670581-ccae-4696-a4fa-cfbf3650826f service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 11:59:58.335031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d670581-ccae-4696-a4fa-cfbf3650826f service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 11:59:58.336628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d670581-ccae-4696-a4fa-cfbf3650826f service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 11:59:58.345613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1169/2320] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 11:59:58 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 12:00:08.271210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:08.341878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=6 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:17.359608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:17.360642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:17.361122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:17.380385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] MFA Rules not processed for user `6cdd0a7f3f12414ea85ebffc9d531773`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:17.381846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:17.443856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1152/2321] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:17 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:00:21.097181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:21.097685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:21.097777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:21.115520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:21.116755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:21.124547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1170/2322] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 12:00:21 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:21.136515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:21.148189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b254c9a7-c8bc-4a84-9b55-617c786ed4f0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:21.153403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:21.154042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:21.154379 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:21.154669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:21.162802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:21.164467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:21.173327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1153/2323] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 12:00:21 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 12:00:24.221260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:24.222857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:24.237676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6303bf34-0ef8-4493-9519-5a81e8f40d89 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:24.242381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:24.243451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:24.243616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:24.243932 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] PATH_INFO: `/v3/users/9082354c3ab249f885e129b86799e508` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:24.246210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] RBAC: Authorizing `identity:delete_user(user_id=9082354c3ab249f885e129b86799e508)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:24.248790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:24.435584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9082354c3ab249f885e129b86799e508'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:24.436245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9082354c3ab249f885e129b86799e508'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:24.496291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9082354c3ab249f885e129b86799e508'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:24.520675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9082354c3ab249f885e129b86799e508'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:24.546913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7447449d-4dc2-461c-8a50-1a4b54cd1ff8 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9082354c3ab249f885e129b86799e508'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:24.555674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1171/2324] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:00:24 2022] DELETE /identity/v3/users/9082354c3ab249f885e129b86799e508 => generated 0 bytes in 335 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:24.842759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:24.854748 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb7382b-c3e3-4e17-9a45-11a86141851a service swift] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:24.941190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:24.954325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:24.954325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:24.954325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] PATH_INFO: `/v3/projects/1c8858f51824444782c874f15da7b16b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:24.966687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] RBAC: Authorizing `identity:delete_project(project_id=1c8858f51824444782c874f15da7b16b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:24.970180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.093342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1c8858f51824444782c874f15da7b16b', 'request_id': 'req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:25.105975 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1c8858f51824444782c874f15da7b16b', 'request_id': 'req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:25.178084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1c8858f51824444782c874f15da7b16b', 'request_id': 'req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:25.195114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] The token cache is being invalidate because project 1c8858f51824444782c874f15da7b16b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:00:25.195554 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:25.196488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cb0a62e0-77c8-4d50-aca7-e4ce4630ed9a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:25.206071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.206743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.207268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.209890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1154/2325] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:00:24 2022] DELETE /identity/v3/projects/1c8858f51824444782c874f15da7b16b => generated 0 bytes in 358 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:25.236769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:25.242513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:25.300503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1172/2326] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:00:25.308518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.308817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.309102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.329204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:25.330439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:25.338769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1155/2327] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:25.343165 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.353744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b81396-8c04-425c-81b2-b018760d75eb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:25.358777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:25.359667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.360132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.361006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.361520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:00:25.362050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:25.363572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.381153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1173/2328] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:00:25.382654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.393084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54de8c02-23b1-4227-a0af-85e327579239 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:25.401913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:25.402864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.403257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.403692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.404756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:25.406878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.422742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:25.535167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1156/2329] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:00:25 2022] POST /identity/v3/projects => generated 371 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:00:25.536115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.548387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2644c0d4-0bbd-43c0-8427-d9db9f8ee606 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:25.606877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:25.615881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:25.616570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.616860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.617136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.617823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:25.619745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.728473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1174/2330] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:00:25 2022] POST /identity/v3/users => generated 286 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:00:25.730185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.741337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d1d016-0189-4b2b-8efb-239e6e7a2bb8 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:25.747087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:25.747760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.748627 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.748918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.749656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:25.751275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.762090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1157/2331] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:00:25.766102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.776553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f61eb5cd-1137-4b4a-b700-2d2633a182bd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:25.782351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86b8f797-46e3-4201-b500-f38ca7b7fe96 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:25.784154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8f797-46e3-4201-b500-f38ca7b7fe96 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.784154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8f797-46e3-4201-b500-f38ca7b7fe96 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.784154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86b8f797-46e3-4201-b500-f38ca7b7fe96 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.784574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8f797-46e3-4201-b500-f38ca7b7fe96 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:25.785994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86b8f797-46e3-4201-b500-f38ca7b7fe96 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.796638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1175/2332] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:00:25 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:00:25.800819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.811897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1558aa-b0a2-4d59-8a41-57eed2675e0b admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:25.816642 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:25.817532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.817820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.818100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] PATH_INFO: `/v3/projects/fab61ecc47674e6e961f67c0b412fda6/users/6f81df1b996f4e6a808c83f4eebe586f/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.832957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fab61ecc47674e6e961f67c0b412fda6, user_id=6f81df1b996f4e6a808c83f4eebe586f, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:25.835774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:25.892257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1158/2333] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:00:25 2022] PUT /identity/v3/projects/fab61ecc47674e6e961f67c0b412fda6/users/6f81df1b996f4e6a808c83f4eebe586f/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:25.897340 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c1e0c-063f-4e84-aaa3-8706032cca95 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:25.897637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c1e0c-063f-4e84-aaa3-8706032cca95 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:25.897920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-880c1e0c-063f-4e84-aaa3-8706032cca95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:25.915976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-880c1e0c-063f-4e84-aaa3-8706032cca95 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:25.917816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-880c1e0c-063f-4e84-aaa3-8706032cca95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:25.981190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1176/2334] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:25.990843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:25.993241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:26.052603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62cce87a-d456-414a-953a-0d0edf4d47e9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:26.056348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:26.057000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:26.057291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:26.057570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:26.067092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:26.068566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:26.078688 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1159/2335] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:00:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:00:31.799150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:31.799678 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:31.799915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:31.828843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] MFA Rules not processed for user `6f81df1b996f4e6a808c83f4eebe586f`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:31.830172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:31.897309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1177/2336] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:31 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:31.902281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:31.913294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5f20713-82f0-4ed0-a5e0-6b116fa13732 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:31.918549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:31.919413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:31.919699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:31.920015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:31.920754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:31.922717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:32.009027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1160/2337] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:00:31 2022] POST /identity/v3/projects => generated 371 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:00:32.012597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:32.025754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab2a573-060d-41ed-90a3-75982741985a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:32.082300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:32.082974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:32.083262 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:32.083542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:32.084270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:32.086197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:32.094154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:32.177010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1178/2338] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:00:32 2022] POST /identity/v3/users => generated 292 bytes in 165 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:00:32.180705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:32.191119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0943d2-53d3-4f43-ba94-3fb69988d06d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:32.195689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:32.196391 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:32.196672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:32.196948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:32.197713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:32.199344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:32.209891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1161/2339] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:00:32.214577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:32.225057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41dfeb2f-e875-4308-9700-4e3a8d1d70f8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:32.230430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:32.231328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:32.231612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:32.232376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] PATH_INFO: `/v3/projects/59052ab8010546d2adafb00ae8fdbd6d/users/3d886cbef1ab4bd1a693df0daff30ad0/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:32.248901 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] RBAC: Authorizing `identity:create_grant(project_id=59052ab8010546d2adafb00ae8fdbd6d, user_id=3d886cbef1ab4bd1a693df0daff30ad0, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:32.251543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:32.349750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1179/2340] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:00:32 2022] PUT /identity/v3/projects/59052ab8010546d2adafb00ae8fdbd6d/users/3d886cbef1ab4bd1a693df0daff30ad0/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 136 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:00:32.353919 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:32.364338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6879e3a-bc38-4053-8d95-f6f0e584e451 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:32.422504 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c80eba-b57f-4ce9-8182-51683d1bb322 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:32.423125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c80eba-b57f-4ce9-8182-51683d1bb322 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:32.423414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c80eba-b57f-4ce9-8182-51683d1bb322 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:32.423692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c80eba-b57f-4ce9-8182-51683d1bb322 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:32.424459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c80eba-b57f-4ce9-8182-51683d1bb322 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:32.426098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5c80eba-b57f-4ce9-8182-51683d1bb322 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:32.440172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1162/2341] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:00:32 2022] GET /identity/v3/roles => generated 1420 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:00:32.441624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:32.453230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f62f78ea-5eb0-45ec-a37b-39f343b3182d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:32.458095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:32.459016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:32.459303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:32.459585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] PATH_INFO: `/v3/projects/59052ab8010546d2adafb00ae8fdbd6d/users/3d886cbef1ab4bd1a693df0daff30ad0/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:32.461673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] RBAC: Authorizing `identity:create_grant(project_id=59052ab8010546d2adafb00ae8fdbd6d, user_id=3d886cbef1ab4bd1a693df0daff30ad0, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:32.466799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:32.481057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1180/2342] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:00:32 2022] PUT /identity/v3/projects/59052ab8010546d2adafb00ae8fdbd6d/users/3d886cbef1ab4bd1a693df0daff30ad0/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:00:38.802980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90748f76-3dde-435c-a8b5-3b62a147a9c9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:38.803640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90748f76-3dde-435c-a8b5-3b62a147a9c9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:38.803640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90748f76-3dde-435c-a8b5-3b62a147a9c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:38.832597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-90748f76-3dde-435c-a8b5-3b62a147a9c9 None None] MFA Rules not processed for user `3d886cbef1ab4bd1a693df0daff30ad0`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:38.833822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-90748f76-3dde-435c-a8b5-3b62a147a9c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:38.854941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=3,keystone:SELECT=97 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:38.920547 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1163/2343] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:38 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:38.934991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:38.936957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:39.024895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253b32c2-dc97-43e6-ae98-6d2dd4d72035 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:39.031186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0658cb5-ec20-40f6-919c-375dceb95378 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:39.031844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658cb5-ec20-40f6-919c-375dceb95378 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:39.032531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658cb5-ec20-40f6-919c-375dceb95378 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:39.032531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0658cb5-ec20-40f6-919c-375dceb95378 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:39.097904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658cb5-ec20-40f6-919c-375dceb95378 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:39.099567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0658cb5-ec20-40f6-919c-375dceb95378 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:39.108646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1181/2344] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:00:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:00:42.399666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:42.400135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:42.400239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:42.447142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] MFA Rules not processed for user `6f81df1b996f4e6a808c83f4eebe586f`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:42.450915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:42.477303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1164/2345] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:42 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 75 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:43.164635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611e7c-a018-4b93-b5ba-79cfbb82ab93 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:43.164635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611e7c-a018-4b93-b5ba-79cfbb82ab93 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:43.165155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52611e7c-a018-4b93-b5ba-79cfbb82ab93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:43.188052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-52611e7c-a018-4b93-b5ba-79cfbb82ab93 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:43.195712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-52611e7c-a018-4b93-b5ba-79cfbb82ab93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:43.272046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:43.284057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1182/2346] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 12:00:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:43.297945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:43.299598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:43.380259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f6c4cad-e6ba-4412-a537-74b3d84fc188 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:43.389148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:43.390132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:43.390601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:43.391117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:43.402101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:43.409899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:43.419935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1165/2347] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 12:00:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 12:00:47.465818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:47.475730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:47.475730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:47.490888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] MFA Rules not processed for user `97ce6d944eb74b9794d6d0d5f18d4bad`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:47.492456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:47.552512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1183/2348] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:53.413086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=58 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:53.459030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:53.460552 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:53.530970 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b398ae-d957-4762-bf55-8e074a1a1868 service swift] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:53.543363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:53.553118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:53.553118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:53.553118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] PATH_INFO: `/v3/users/e6730f4274e240dd9da30e5c3bc3679b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:53.553118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=e6730f4274e240dd9da30e5c3bc3679b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:53.553118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:53.708033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e6730f4274e240dd9da30e5c3bc3679b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:53.708691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e6730f4274e240dd9da30e5c3bc3679b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:53.771906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e6730f4274e240dd9da30e5c3bc3679b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:53.780732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e6730f4274e240dd9da30e5c3bc3679b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:53.794441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e6730f4274e240dd9da30e5c3bc3679b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:53.800531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1166/2349] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:00:53 2022] DELETE /identity/v3/users/e6730f4274e240dd9da30e5c3bc3679b => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:54.059638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:54.071441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9817da72-8bc1-406d-85c5-8a003f011b64 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:54.131490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:54.132490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:54.132663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:54.132939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] PATH_INFO: `/v3/projects/345dc1b6fec149e88f1be7837d126d20` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:54.134243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] RBAC: Authorizing `identity:delete_project(project_id=345dc1b6fec149e88f1be7837d126d20)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:54.136718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:54.269055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '345dc1b6fec149e88f1be7837d126d20', 'request_id': 'req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.275621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '345dc1b6fec149e88f1be7837d126d20', 'request_id': 'req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.282949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '345dc1b6fec149e88f1be7837d126d20', 'request_id': 'req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.389016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] The token cache is being invalidate because project 345dc1b6fec149e88f1be7837d126d20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:00:54.389444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.390285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.391920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1184/2350] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:00:54 2022] DELETE /identity/v3/projects/345dc1b6fec149e88f1be7837d126d20 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:54.397611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:54.398773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:54.412808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87934dcc-a123-46ae-9f2f-287fd9dd4b4f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:54.484204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:54.485157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:54.485448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:54.485717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] PATH_INFO: `/v3/users/21d97de7b77d4cd096f00c175c4a3556` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:54.486976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] RBAC: Authorizing `identity:delete_user(user_id=21d97de7b77d4cd096f00c175c4a3556)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:54.488973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:54.511599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:54.698262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '21d97de7b77d4cd096f00c175c4a3556'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.700062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '21d97de7b77d4cd096f00c175c4a3556'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.789364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '21d97de7b77d4cd096f00c175c4a3556'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.797454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '21d97de7b77d4cd096f00c175c4a3556'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.811400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5388cfcc-88b3-460d-aa82-9a83d637cf8d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '21d97de7b77d4cd096f00c175c4a3556'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:54.821803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1167/2351] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:00:54 2022] DELETE /identity/v3/users/21d97de7b77d4cd096f00c175c4a3556 => generated 0 bytes in 423 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:55.087752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:55.089325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:55.103524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2c77c7c-038a-4cc5-95b9-0dcd0529a8ee admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:55.152874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:55.180928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:55.181787 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.182155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.182343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] PATH_INFO: `/v3/projects/bd0df547d2734080a1fecaf71ae4b520` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.183701 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bd0df547d2734080a1fecaf71ae4b520)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:55.185496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:55.346055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bd0df547d2734080a1fecaf71ae4b520', 'request_id': 'req-234a1bc9-a99a-4e11-9b45-e903fbbe937a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:55.359669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bd0df547d2734080a1fecaf71ae4b520', 'request_id': 'req-234a1bc9-a99a-4e11-9b45-e903fbbe937a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:55.368087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bd0df547d2734080a1fecaf71ae4b520', 'request_id': 'req-234a1bc9-a99a-4e11-9b45-e903fbbe937a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:55.451246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] The token cache is being invalidate because project bd0df547d2734080a1fecaf71ae4b520 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:00:55.451542 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:55.452185 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-234a1bc9-a99a-4e11-9b45-e903fbbe937a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:00:55.453725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1185/2352] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:00:55 2022] DELETE /identity/v3/projects/bd0df547d2734080a1fecaf71ae4b520 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:55.458696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.458990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.459275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.478824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:55.480718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:55.540115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1168/2353] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:55.547767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.548107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.548419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.567213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:55.568611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:55.577273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1186/2354] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:55.581631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:55.595742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff78881-88cd-486a-b70b-23da8169c157 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:55.603644 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:55.605207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.605657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.606091 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.607143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:00:55.607628 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:55.609840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:55.631263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1169/2355] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:00:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 50 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:00:55.635282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:55.654040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b0d5f98-3ebd-41c5-8c54-7afbe4cfe22c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:55.659357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:55.660185 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.660939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.661229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.661935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:55.664523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:55.784633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1187/2356] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:00:55 2022] POST /identity/v3/projects => generated 377 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:00:55.789054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:55.799221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-10f40cb8-92ba-4d74-88f5-40615977ed03 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:55.863193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:55.863824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.864725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.865017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.865842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:55.868002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:55.938996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1170/2357] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:00:55 2022] POST /identity/v3/users => generated 289 bytes in 150 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:00:55.942926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:55.955468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da8c01e1-428f-4172-9355-4582a041eef4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:55.960583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:55.961253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.961541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.961848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.962581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:55.964236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:55.976271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1188/2358] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:00:55.980418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:55.991241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23d767c6-a2ce-4349-8b88-fc4f0b41d02d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:55.996532 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64b01675-0677-4382-8ff1-b6a7bfe897c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:55.997229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b01675-0677-4382-8ff1-b6a7bfe897c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:55.997511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b01675-0677-4382-8ff1-b6a7bfe897c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:55.997781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64b01675-0677-4382-8ff1-b6a7bfe897c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:55.998722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b01675-0677-4382-8ff1-b6a7bfe897c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:56.000915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64b01675-0677-4382-8ff1-b6a7bfe897c7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:56.018403 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1171/2359] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:00:55 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:00:56.019687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:56.030109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c92ef6c6-8a77-437f-9fdb-166ba3153f51 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:56.036469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:56.037979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:56.038275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:56.038559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] PATH_INFO: `/v3/projects/4be7fa7276cb42d3a8b6dd0dd12cd784/users/f6e726651e4c49b794f5c86e1fe51d9c/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:56.057181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4be7fa7276cb42d3a8b6dd0dd12cd784, user_id=f6e726651e4c49b794f5c86e1fe51d9c, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:56.060455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:56.144127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1189/2360] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:00:56 2022] PUT /identity/v3/projects/4be7fa7276cb42d3a8b6dd0dd12cd784/users/f6e726651e4c49b794f5c86e1fe51d9c/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 125 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:00:56.149544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:56.155414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:56.155414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:56.157435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=7,keystone:SELECT=89,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:56.168440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:00:56.169746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:56.236480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1172/2361] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:00:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:00:56.247043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:56.249583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:56.322435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598f3db6-fca3-410d-919b-5b6fe0201c00 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:56.326305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:56.326967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:56.327568 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:56.327861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:56.337743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:56.339261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:56.344740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:00:56.350375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1190/2362] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:00:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:00:58.347128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:58.348672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:58.366850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80df8fd3-2c7c-4fd0-9844-16d79575c8bf None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:58.371440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:58.372276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:58.372959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:58.377654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:58.383744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:58.475328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:58.476984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56f71dcf-3829-4db1-a98e-17c76fe66bcb service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:58.490021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1173/2363] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:00:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 143 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:00:59.740791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:00:59.749969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a17c20af-37bd-40e3-8826-b0d14f489e1e service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:00:59.753753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:00:59.754411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:00:59.754698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:00:59.754980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:00:59.756798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:00:59.772629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:00:59.774082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c7da0f-9045-4746-90d8-d84bf9e39030 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:00:59.787521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1191/2364] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:00:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:00.795066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:00.795907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:00.796370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:00.819384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] MFA Rules not processed for user `5e69fc12fcc34fa39f67dd156cd860a4`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:00.821326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:00.907299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1174/2365] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:00 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:01.989123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:01.989714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:01.989714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.019704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] MFA Rules not processed for user `f6e726651e4c49b794f5c86e1fe51d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:02.021350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:02.104320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1192/2366] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:01 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:02.109448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:02.122115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b54e0e-7085-4e46-a75b-9eca437f232d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:02.127668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:02.128771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:02.129209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:02.129663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.131231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:02.133790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:02.220034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1175/2367] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:01:02 2022] POST /identity/v3/projects => generated 379 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:01:02.224420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:02.237308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cf8ab55-32a3-4ff9-b4f9-4dd06c8c882f None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:02.356916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:02.357704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:02.358753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:02.359213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.360639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:02.370983 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:02.474431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1193/2368] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:01:02 2022] POST /identity/v3/users => generated 296 bytes in 248 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:01:02.475765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:02.487416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c903f71-7ffc-408c-9ccd-5ba61d11c216 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:02.492392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:02.493062 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:02.493347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:02.493621 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.495045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:02.499045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:02.510754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1176/2369] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:02.515030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:02.526065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c765af3d-65dc-4b73-9548-40d1ecfebd86 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:02.530977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:02.531891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:02.532178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:02.532459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] PATH_INFO: `/v3/projects/3be18f93e33a4a0cb91c958bf8bbcd4b/users/7089d744edef40179e8bdf3b304fce5f/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.551352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be18f93e33a4a0cb91c958bf8bbcd4b, user_id=7089d744edef40179e8bdf3b304fce5f, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:02.554293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:02.631229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1194/2370] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:01:02 2022] PUT /identity/v3/projects/3be18f93e33a4a0cb91c958bf8bbcd4b/users/7089d744edef40179e8bdf3b304fce5f/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:02.633142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:02.638839 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:02.646182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8668eb12-230b-499d-971d-b63e776f9b3e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:02.727820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bfc84ff-74f1-46fa-b64c-23b9fc470f83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:02.732104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc84ff-74f1-46fa-b64c-23b9fc470f83 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:02.732574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc84ff-74f1-46fa-b64c-23b9fc470f83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:02.733014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bfc84ff-74f1-46fa-b64c-23b9fc470f83 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.734747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc84ff-74f1-46fa-b64c-23b9fc470f83 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:02.736750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bfc84ff-74f1-46fa-b64c-23b9fc470f83 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:02.751080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1177/2371] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:02 2022] GET /identity/v3/roles => generated 1420 bytes in 118 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:02.752207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:02.766251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da9058a0-cdbe-4b01-95bd-1202033ed07b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:02.782679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:02.784369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:02.784671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:02.784959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] PATH_INFO: `/v3/projects/3be18f93e33a4a0cb91c958bf8bbcd4b/users/7089d744edef40179e8bdf3b304fce5f/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:02.787040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] RBAC: Authorizing `identity:create_grant(project_id=3be18f93e33a4a0cb91c958bf8bbcd4b, user_id=7089d744edef40179e8bdf3b304fce5f, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:02.790361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:02.805499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1195/2372] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:01:02 2022] PUT /identity/v3/projects/3be18f93e33a4a0cb91c958bf8bbcd4b/users/7089d744edef40179e8bdf3b304fce5f/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:12.767304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:12.767915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:12.768294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:12.803019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=82,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:12.831790 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] MFA Rules not processed for user `7089d744edef40179e8bdf3b304fce5f`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:12.834206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:12.992745 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1178/2373] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:12 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 227 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:13.008412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:13.010422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:13.030529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:13.159613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd35a1d-ef0a-4bd2-842c-f7b96f3e0a71 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:13.168887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e89867-f74a-4cfa-bd07-4fe4da06814d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:13.171402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:13.172100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:13.172387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:13.172655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:13.177692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:13.178666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:13.179162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:13.179692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:13.185407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:13.275693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:13.278158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:13.296248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1196/2374] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 288 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:13.298043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:13.342165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:13.343795 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb3988af-0be2-442c-a3fc-bf4f0ff4730e service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:13.366105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1179/2375] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:13 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:13.946013 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:13.954298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28c81398-c2fb-4a46-8793-6f29fefb196c service nova] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:13.958045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:13.958689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:13.958980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:13.959263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:13.961216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:14.057427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:14.059150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:14.073824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1197/2376] 10.222.0.40 () {60 vars in 1396 bytes} [Thu Jun 16 12:01:13 2022] GET /identity/v3/auth/tokens?allow_expired=1 => generated 3260 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:01:18.133757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:18.133757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:18.133757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:18.163220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] MFA Rules not processed for user `13d8c04611e24dffad16f8bb97e111f2`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:18.164738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:18.178791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1180/2377] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:18 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:18.762407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:18.773922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fa9ca2-2485-4985-bce4-2a1b4e6152bf service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:18.782641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:18.782641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:18.782641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:18.782641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:18.783617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:18.794444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:18.796167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:18.806121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1198/2378] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 44 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:19.329429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:19.337958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:19.444083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d8d451c-cf78-4f6c-a6c4-c73808f26ad1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:19.450386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:19.451137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:19.451496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:19.451893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:19.453849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:19.466289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:19.472550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:19.485818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1181/2379] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:27.005473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:27.014967 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24f75d25-afb4-439a-9977-f335a3b4ad5c service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:27.018659 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:27.019307 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:27.019594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:27.019925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:27.021691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:27.032338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:27.033858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:27.037133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:27.043462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1199/2380] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:27.047160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b5a647c-4ab5-484b-be41-efd2c8bb3041 service placement] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:27.050867 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:27.051518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:27.051807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:27.052138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:27.060357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:27.061813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:27.070159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1182/2381] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:27 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:01:30.672332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:30.681460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8c198f5-87e7-497e-a754-8cfbf2327d1a service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:30.685989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:30.686635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:30.686930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:30.687211 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:30.689150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:30.695729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:30.760847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:30.762379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a31dcf-331c-4411-8aba-580a01c617db service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:30.773027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1200/2382] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:30 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 101 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:36.845543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:36.847251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:36.863424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2318a87-054d-43fe-8c13-609113a7dd91 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:36.868894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:36.877464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:36.877464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:36.877464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:36.877464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:36.884346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:36.886492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:36.897381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1183/2383] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:36 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:37.369182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:37.369779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:37.369779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:37.390362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] MFA Rules not processed for user `f6e726651e4c49b794f5c86e1fe51d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:37.391566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:37.399140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1201/2384] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:37 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:37.719710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:37.732179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4220d58c-a6a7-4bef-8123-17375edaaa34 service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:37.736198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:37.736983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:37.737274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:37.737553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:37.739320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:37.749814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:37.751241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:37.759640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1184/2385] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:37 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:42.173220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:42.183550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceab7ee4-73a4-4452-9695-3a8ed4957ea3 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:42.187430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:42.188337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:42.188627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:42.189324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:42.191158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:42.258204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:42.259665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:42.271047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1202/2386] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:01:46.574736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:46.576258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:46.586213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=64 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:46.595214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1b8a3ae-894e-437e-a529-4ef5008a6bea service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:46.601451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:46.602125 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:46.602416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:46.602703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:46.605099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:46.617999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:46.619637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:46.631051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1185/2387] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:46 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 57 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:46.737753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:46.740305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:46.756887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70f5f575-7877-4f46-bd8f-41872f30bb18 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:46.763508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:46.764762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:46.765060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:46.775451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] PATH_INFO: `/v3/users/6cdd0a7f3f12414ea85ebffc9d531773` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:46.775451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6cdd0a7f3f12414ea85ebffc9d531773)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:46.775451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:46.945536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6cdd0a7f3f12414ea85ebffc9d531773'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:46.946279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6cdd0a7f3f12414ea85ebffc9d531773'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:46.994080 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6cdd0a7f3f12414ea85ebffc9d531773'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.001817 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6cdd0a7f3f12414ea85ebffc9d531773'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.014650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4a4f14b8-60f9-4c62-bb62-9cb0ea270617 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6cdd0a7f3f12414ea85ebffc9d531773'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.020790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1203/2388] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:01:46 2022] DELETE /identity/v3/users/6cdd0a7f3f12414ea85ebffc9d531773 => generated 0 bytes in 284 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:47.265716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:47.279127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2189c53e-64ff-4006-b7c7-b290e6185f58 service placement] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:47.336806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:47.337676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:47.338031 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:47.338315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] PATH_INFO: `/v3/projects/ce5409289ccf4aad8d62b13a72d0009e` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:47.339498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ce5409289ccf4aad8d62b13a72d0009e)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:47.341445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:47.490586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ce5409289ccf4aad8d62b13a72d0009e', 'request_id': 'req-71873827-92dc-4706-977d-204befa13e31'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.496566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ce5409289ccf4aad8d62b13a72d0009e', 'request_id': 'req-71873827-92dc-4706-977d-204befa13e31'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.559382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ce5409289ccf4aad8d62b13a72d0009e', 'request_id': 'req-71873827-92dc-4706-977d-204befa13e31'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.570743 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] The token cache is being invalidate because project ce5409289ccf4aad8d62b13a72d0009e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:01:47.575153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.575153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-71873827-92dc-4706-977d-204befa13e31 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:47.575153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1186/2389] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:01:47 2022] DELETE /identity/v3/projects/ce5409289ccf4aad8d62b13a72d0009e => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:47.578868 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:47.579167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:47.579456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:47.585857 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:47.598347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:47.603914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:47.658705 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1204/2390] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:47.666364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:47.667370 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:47.667655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:47.686513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:47.687723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:47.695660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1187/2391] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:47.699993 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:47.709894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4aac85-f8e4-4b21-94be-e45f7f004263 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:47.714666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:47.715326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:47.715615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:47.715931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:47.716665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:01:47.716980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:47.718653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:47.731674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1205/2392] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:01:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:01:47.735766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:47.745466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1e964f-0ccc-4122-9c64-319285460367 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:47.750161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:47.750808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:47.751095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:47.751372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:47.752090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:47.754026 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:47.844505 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1188/2393] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:01:47 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:01:47.845878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:47.859005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86e24f51-0b64-4eae-b750-8ad91e416f20 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:47.933002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:47.933002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:47.933002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:47.933002 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:47.933424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:47.933424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:48.019903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1206/2394] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:01:47 2022] POST /identity/v3/users => generated 292 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:01:48.021710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:48.037687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d9289-93c4-4df6-b9ab-7e9d84bde8a0 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:48.037810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:48.038479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:48.038843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:48.039124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:48.039858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:48.041539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:48.052945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1189/2395] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:48.056979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:48.073091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b212e6b6-00d3-4a88-a02f-1e3fb3ec186f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:48.077979 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cc689db-1bb6-42f3-bb17-deb4f7ac4703 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:48.078695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc689db-1bb6-42f3-bb17-deb4f7ac4703 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:48.078985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc689db-1bb6-42f3-bb17-deb4f7ac4703 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:48.079268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cc689db-1bb6-42f3-bb17-deb4f7ac4703 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:48.080119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc689db-1bb6-42f3-bb17-deb4f7ac4703 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:48.082115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7cc689db-1bb6-42f3-bb17-deb4f7ac4703 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:48.093367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1207/2396] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:48 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:48.097890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:48.112235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-841e5655-200b-48d9-aef1-0f7ded6d0d9a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:48.117118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:48.118038 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:48.118296 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:48.118570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] PATH_INFO: `/v3/projects/fd9a1d686d0f4a59817e2a94e58ab120/users/a28ec3b323ae437b9154a6ba98d8439c/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:48.134372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd9a1d686d0f4a59817e2a94e58ab120, user_id=a28ec3b323ae437b9154a6ba98d8439c, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:48.137206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:48.230568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1190/2397] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:01:48 2022] PUT /identity/v3/projects/fd9a1d686d0f4a59817e2a94e58ab120/users/a28ec3b323ae437b9154a6ba98d8439c/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 133 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:48.245067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:48.245361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:48.245669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:48.272576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] MFA Rules not processed for user `a28ec3b323ae437b9154a6ba98d8439c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:48.273823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:48.335220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:48.341299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1208/2398] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:48 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:01:48.352728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:48.355242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:48.373779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:48.429277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e23c3cc-98b4-4920-bfbb-950dade170b4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:48.434846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-487724fd-1a8d-4023-b751-051150e73c9b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:48.435493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487724fd-1a8d-4023-b751-051150e73c9b service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:48.435776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487724fd-1a8d-4023-b751-051150e73c9b service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:48.436084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487724fd-1a8d-4023-b751-051150e73c9b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:48.444284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487724fd-1a8d-4023-b751-051150e73c9b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:48.445875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-487724fd-1a8d-4023-b751-051150e73c9b service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:48.455075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1191/2399] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:48 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:56.525622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:56.585654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03300e6b-56ac-4cf5-a65e-14a1b3469a01 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:56.590105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:56.590997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:56.591330 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:56.591604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] PATH_INFO: `/v3/users/a28ec3b323ae437b9154a6ba98d8439c` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:56.592811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] RBAC: Authorizing `identity:delete_user(user_id=a28ec3b323ae437b9154a6ba98d8439c)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:56.594726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:56.790079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a28ec3b323ae437b9154a6ba98d8439c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:56.790747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a28ec3b323ae437b9154a6ba98d8439c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:56.850891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a28ec3b323ae437b9154a6ba98d8439c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:56.858482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a28ec3b323ae437b9154a6ba98d8439c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:56.871741 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a28ec3b323ae437b9154a6ba98d8439c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:56.877824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1209/2400] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:01:56 2022] DELETE /identity/v3/users/a28ec3b323ae437b9154a6ba98d8439c => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:56.882415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:56.882865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:56.883005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:56.899964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:56.901222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:56.960225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1192/2401] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:56.970062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:56.971844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:57.038534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4adc1d85-1d5d-492e-94ca-31e6544132fd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:57.043249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:57.044073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:57.044366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:57.044647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:57.055429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:57.056944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:57.066919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1210/2402] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:01:57.092847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:57.095040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:57.171126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feae3e24-6a77-497c-8daf-6c51aa1242a7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:57.176734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:57.177393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:57.177677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:57.177955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:57.179714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:57.191887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:57.193414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc91da94-5483-4d95-a632-9076d7d8ea3d service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:57.205053 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1193/2403] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:57 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:57.591771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:57.593985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:57.607506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-543c2efe-e5c1-4e36-9df7-d16f4265aae1 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:57.617841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:57.619266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:57.619266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:57.619812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] PATH_INFO: `/v3/projects/fd9a1d686d0f4a59817e2a94e58ab120` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:57.621167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fd9a1d686d0f4a59817e2a94e58ab120)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:57.623092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:57.647922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:57.754101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fd9a1d686d0f4a59817e2a94e58ab120', 'request_id': 'req-34742fb1-9906-4802-aed3-ed1ccd08b874'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:57.760786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fd9a1d686d0f4a59817e2a94e58ab120', 'request_id': 'req-34742fb1-9906-4802-aed3-ed1ccd08b874'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:57.766216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fd9a1d686d0f4a59817e2a94e58ab120', 'request_id': 'req-34742fb1-9906-4802-aed3-ed1ccd08b874'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:57.867582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] The token cache is being invalidate because project fd9a1d686d0f4a59817e2a94e58ab120 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:01:57.867896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:57.868523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-34742fb1-9906-4802-aed3-ed1ccd08b874 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:01:57.870559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1211/2404] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:01:57 2022] DELETE /identity/v3/projects/fd9a1d686d0f4a59817e2a94e58ab120 => generated 0 bytes in 279 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:57.875413 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:57.875710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:57.876028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:57.895067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:57.896623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:57.916511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:57.954257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1194/2405] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:57.962902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:57.964439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:57.964439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:57.989312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:57.989312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:57.991397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1212/2406] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:57.996119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.005697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4358d7f9-1f24-4250-99d5-504e118f2a97 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.010915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.011568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.011859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.012190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.012909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:01:58.013224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.014824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.026164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1195/2407] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:01:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:01:58.030268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.039420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42e2c682-c512-426e-9cc3-3018fb3dfad2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.044191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.044849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.045137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.045421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.046123 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.048101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.066393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.068112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:58.130572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1213/2408] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:01:58 2022] POST /identity/v3/projects => generated 365 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:01:58.131509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.143033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2afb032-7cde-41ac-a81e-890642f9f0f7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.207158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c7fe87e-34bf-41b9-9c2c-3a4406e88f32 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.211860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.212564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.212856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.213055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.213187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.213721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.213862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.214005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.214278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.215800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.216079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:58.264850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:58.275097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.276776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.288817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1196/2410] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:58 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 220 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:01:58.289335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1214/2410] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:01:58 2022] POST /identity/v3/users => generated 283 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:01:58.289894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.300371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9f6c8d3-eb3e-46e7-98e9-a592e9dd740f service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.305708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.306376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.306663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.307586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.308349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.309940 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.319791 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1197/2411] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:58.323985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.334540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0167dbb6-431a-4d2e-930d-466a98e55fe2 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.339358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-887cc168-a4dd-4e63-8d7a-b13154f09a7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.340076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cc168-a4dd-4e63-8d7a-b13154f09a7d admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.340362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cc168-a4dd-4e63-8d7a-b13154f09a7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.340644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-887cc168-a4dd-4e63-8d7a-b13154f09a7d admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.341404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cc168-a4dd-4e63-8d7a-b13154f09a7d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.343037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-887cc168-a4dd-4e63-8d7a-b13154f09a7d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.357541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1215/2412] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:58.358522 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.368722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de762bef-4deb-4057-a5de-ac101a6c8900 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.373851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.374764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.375048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.375327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] PATH_INFO: `/v3/projects/cba89085f8f64f2995708a22b01d1c4a/users/9036f8c12a3b4f6cb28339e6dcfbf31c/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.390249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cba89085f8f64f2995708a22b01d1c4a, user_id=9036f8c12a3b4f6cb28339e6dcfbf31c, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.392958 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.461401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1198/2413] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:01:58 2022] PUT /identity/v3/projects/cba89085f8f64f2995708a22b01d1c4a/users/9036f8c12a3b4f6cb28339e6dcfbf31c/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:01:58.471830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182433b7-a559-4010-8111-8da755628538 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.472158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182433b7-a559-4010-8111-8da755628538 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.472441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-182433b7-a559-4010-8111-8da755628538 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.500321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-182433b7-a559-4010-8111-8da755628538 None None] MFA Rules not processed for user `9036f8c12a3b4f6cb28339e6dcfbf31c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:58.501545 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-182433b7-a559-4010-8111-8da755628538 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:58.545420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=88,keystone:DELETE=7,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:58.571471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1216/2414] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:58 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:58.576184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.643352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a53aa0a-c6f8-47a5-af65-face10cd3636 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.648942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.649674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.652557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.652557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.652557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.652928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.743992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1199/2415] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:01:58 2022] POST /identity/v3/projects => generated 365 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:01:58.748583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182433b7-a559-4010-8111-8da755628538 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.759287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-182433b7-a559-4010-8111-8da755628538 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.827384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.829607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.829607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.829607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.829607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.831261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.908826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1217/2416] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:01:58 2022] POST /identity/v3/users => generated 289 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:01:58.912787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.923583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b432b3d9-19aa-4f4c-b54a-471f76bf551c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.928383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.933089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.933089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.933089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.933089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.933703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:58.944514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1200/2417] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:58 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:58.948715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:58.958804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d3a067-9b9c-4dcb-9b66-3d1938e5ed68 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:58.963465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:58.964384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:58.964675 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:58.964956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] PATH_INFO: `/v3/projects/dd7fe42dc8ef4d0a8d86d2408f3a3925/users/d406b254307e4d7684113fbb2c224bef/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:58.980388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7fe42dc8ef4d0a8d86d2408f3a3925, user_id=d406b254307e4d7684113fbb2c224bef, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:58.983116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:59.086247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1218/2418] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:01:58 2022] PUT /identity/v3/projects/dd7fe42dc8ef4d0a8d86d2408f3a3925/users/d406b254307e4d7684113fbb2c224bef/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:59.090204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:59.100439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1f95f28-09c8-461f-9ae4-1be5729552c9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:59.156423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74a275ad-5714-4b3c-87bc-7b436de9d770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:59.157120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a275ad-5714-4b3c-87bc-7b436de9d770 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:59.157471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a275ad-5714-4b3c-87bc-7b436de9d770 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:59.157753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74a275ad-5714-4b3c-87bc-7b436de9d770 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:59.158475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a275ad-5714-4b3c-87bc-7b436de9d770 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:59.160114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74a275ad-5714-4b3c-87bc-7b436de9d770 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:59.170154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1201/2419] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:01:59 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:01:59.174290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:59.185226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa23d3df-4dc3-4b9d-b964-b7ec4d9d086b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:59.189796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:59.190670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:59.190954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:59.191243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] PATH_INFO: `/v3/projects/dd7fe42dc8ef4d0a8d86d2408f3a3925/users/d406b254307e4d7684113fbb2c224bef/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:59.193203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dd7fe42dc8ef4d0a8d86d2408f3a3925, user_id=d406b254307e4d7684113fbb2c224bef, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:59.195969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:59.206083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1219/2420] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:01:59 2022] PUT /identity/v3/projects/dd7fe42dc8ef4d0a8d86d2408f3a3925/users/d406b254307e4d7684113fbb2c224bef/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:01:59.216322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:59.216617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:59.216922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:59.243399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] MFA Rules not processed for user `d406b254307e4d7684113fbb2c224bef`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:01:59.245313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:59.256947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:01:59.313354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1202/2421] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:01:59 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:01:59.326485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:59.328193 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:59.386658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0e466b7-2171-4ab6-8b9f-743c3258e281 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:59.390216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:59.390854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:59.391137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:59.391415 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:59.400312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:59.402269 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:59.411616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1220/2422] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:59 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:01:59.607353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:01:59.608810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:01:59.621143 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b74a540-0998-45b9-a13f-b0df66077b90 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:01:59.624689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:01:59.625316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:01:59.625600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:01:59.625877 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:01:59.682406 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:01:59.684045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:01:59.694421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1203/2423] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:01:59 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:02.386625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:02.388968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:02.437182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:02.451978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42dce283-62d5-475d-97f5-cccdba792c93 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:02.455583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:02.456267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:02.456583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:02.456867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:02.458638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:02.518804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:02.520316 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:02.530195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1221/2424] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:02 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 144 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:02:04.729672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:04.805498 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a982c7e7-02a0-4e4c-ab89-fab78f746022 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:04.811393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:04.812084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:04.812399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:04.812675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:04.814462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:04.827207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:04.828862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5485cc6-2510-4f4b-b950-94cba47b407a service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:04.840372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1204/2425] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:04 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:12.901277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=33 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:13.501397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:13.565082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4934e8-ec11-42d1-a3b2-60ebda8c1ebf service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:13.569616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:13.570494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:13.570782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:13.571063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] PATH_INFO: `/v3/users/9036f8c12a3b4f6cb28339e6dcfbf31c` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:13.572291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] RBAC: Authorizing `identity:delete_user(user_id=9036f8c12a3b4f6cb28339e6dcfbf31c)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:13.574297 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:13.777935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9036f8c12a3b4f6cb28339e6dcfbf31c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:13.778607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9036f8c12a3b4f6cb28339e6dcfbf31c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:13.866405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9036f8c12a3b4f6cb28339e6dcfbf31c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:13.873910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9036f8c12a3b4f6cb28339e6dcfbf31c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:13.886476 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9036f8c12a3b4f6cb28339e6dcfbf31c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:13.892418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1222/2426] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:13 2022] DELETE /identity/v3/users/9036f8c12a3b4f6cb28339e6dcfbf31c => generated 0 bytes in 392 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:13.896868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:13.897347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:13.897480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:13.920654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:13.921923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:13.951481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:13.982339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1205/2427] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:13.991673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:13.993175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:14.049448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04fb986a-fdc8-4a2b-ab57-65ad5fbfcacc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:14.053044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:14.053686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:14.053973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:14.054252 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:14.062980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:14.064444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:14.074540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1223/2428] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:14.478336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:14.479841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:14.494508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86183d21-23c8-447c-85bc-25c5e6922ff9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:14.500691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:14.501643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:14.502010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:14.502356 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] PATH_INFO: `/v3/projects/cba89085f8f64f2995708a22b01d1c4a` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:14.504220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cba89085f8f64f2995708a22b01d1c4a)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:14.506741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:14.611421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cba89085f8f64f2995708a22b01d1c4a', 'request_id': 'req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:14.616788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cba89085f8f64f2995708a22b01d1c4a', 'request_id': 'req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:14.702344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cba89085f8f64f2995708a22b01d1c4a', 'request_id': 'req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:14.714431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] The token cache is being invalidate because project cba89085f8f64f2995708a22b01d1c4a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:14.714722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:14.715378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:14.719099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1206/2429] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:14 2022] DELETE /identity/v3/projects/cba89085f8f64f2995708a22b01d1c4a => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:14.725219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:14.736295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95adbe88-48db-4548-a194-e5a9b87baa3b service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:14.757338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:14.796030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:14.796909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:14.797195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:14.797475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] PATH_INFO: `/v3/users/d406b254307e4d7684113fbb2c224bef` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:14.798750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d406b254307e4d7684113fbb2c224bef)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:14.800691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:15.007451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd406b254307e4d7684113fbb2c224bef'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.008125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd406b254307e4d7684113fbb2c224bef'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.090576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd406b254307e4d7684113fbb2c224bef'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.098550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd406b254307e4d7684113fbb2c224bef'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.111966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1ff4542b-9905-47c4-bfed-935e745e45b7 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd406b254307e4d7684113fbb2c224bef'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.117957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1224/2430] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:14 2022] DELETE /identity/v3/users/d406b254307e4d7684113fbb2c224bef => generated 0 bytes in 393 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:15.598978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:15.600610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:15.614906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6cc996f-1b63-4f2c-b2cc-32d86e8457b4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:15.677602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:15.678579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:15.678962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:15.679432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] PATH_INFO: `/v3/projects/dd7fe42dc8ef4d0a8d86d2408f3a3925` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:15.680773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dd7fe42dc8ef4d0a8d86d2408f3a3925)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:15.682767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:15.788475 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dd7fe42dc8ef4d0a8d86d2408f3a3925', 'request_id': 'req-cfdc8cc7-8e06-4793-a6ae-4f015429a334'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.794459 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dd7fe42dc8ef4d0a8d86d2408f3a3925', 'request_id': 'req-cfdc8cc7-8e06-4793-a6ae-4f015429a334'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.796420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:15.879647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dd7fe42dc8ef4d0a8d86d2408f3a3925', 'request_id': 'req-cfdc8cc7-8e06-4793-a6ae-4f015429a334'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.893988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] The token cache is being invalidate because project dd7fe42dc8ef4d0a8d86d2408f3a3925 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:15.894278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.894998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-cfdc8cc7-8e06-4793-a6ae-4f015429a334 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:15.896653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1207/2431] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:15 2022] DELETE /identity/v3/projects/dd7fe42dc8ef4d0a8d86d2408f3a3925 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:15.901923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:15.902216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:15.902499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:15.920537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:15.921727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:15.987018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1225/2432] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:15.995315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:15.995608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:15.995916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.014483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:16.015721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:16.023751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1208/2433] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:02:16.027862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.037399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0175363f-a46c-4e39-bf5b-8adddac72d39 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.041907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.042562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.042848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.043126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.043850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:16.044187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.045777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.057198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1226/2434] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:02:16.061813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.070708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e9f9ce6-8853-4691-a7be-20e75734dc80 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.075964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.076635 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.076894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.077169 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.077858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.079738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.310559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1209/2435] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:16 2022] POST /identity/v3/projects => generated 379 bytes in 249 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:02:16.314813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.325100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f06ee9e-234e-44de-9350-28fd1a944ac2 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.351200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:16.382922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.383594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.383907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.384196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.384892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.386855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.472549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1227/2436] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:16 2022] POST /identity/v3/users => generated 290 bytes in 158 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:16.476660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.487580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84370c68-50d4-4fe5-bd90-f7e1629f0d37 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.492227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.492886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.493170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.493445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.494267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.496550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.507277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1210/2437] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:16.511171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.520878 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6cb641d-d729-4e77-9282-2d1ccb6d7fcc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.525478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da63e32-2634-48b5-b349-86aeae5c0f13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.526673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da63e32-2634-48b5-b349-86aeae5c0f13 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.526964 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da63e32-2634-48b5-b349-86aeae5c0f13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.527244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da63e32-2634-48b5-b349-86aeae5c0f13 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.528011 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da63e32-2634-48b5-b349-86aeae5c0f13 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.529600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da63e32-2634-48b5-b349-86aeae5c0f13 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.539259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1228/2438] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:16 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:02:16.543862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.553700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-376e32ca-0555-4fee-a965-fb39e23886d4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.558315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.559192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.559479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.559756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] PATH_INFO: `/v3/projects/11f37382fe3c46c988b473d6473949e3/users/dc0e0054c0a144eea24a3b80104718d0/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.574015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=11f37382fe3c46c988b473d6473949e3, user_id=dc0e0054c0a144eea24a3b80104718d0, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.576647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.674488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1211/2439] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:16 2022] PUT /identity/v3/projects/11f37382fe3c46c988b473d6473949e3/users/dc0e0054c0a144eea24a3b80104718d0/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:16.684694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.684991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.685274 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.716026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] MFA Rules not processed for user `dc0e0054c0a144eea24a3b80104718d0`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:16.717259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:16.785903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1229/2440] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:16 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:16.790325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.851835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dee5f197-385e-4227-909f-24ddd33fa8b4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:16.857297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:16.857964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:16.858248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:16.858524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:16.859210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:16.861142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:16.973217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1212/2441] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:16 2022] POST /identity/v3/projects => generated 381 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:16.977168 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:16.988552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84e79599-cee8-4914-a66a-2f8f4654dc34 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:17.044360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:17.045103 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.045300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.045575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.046251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:17.048173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:17.137278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1230/2442] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:16 2022] POST /identity/v3/users => generated 297 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:17.141092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:17.152028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc238bff-8357-4358-bb7f-7bf6ff5ae8d1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:17.156667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:17.159496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.159496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.159496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.159496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:17.160312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:17.170750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1213/2443] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:17.174735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:17.184651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-549c388d-2a5e-4b7a-8f19-68228664e587 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:17.189222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:17.190253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.190543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.190824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] PATH_INFO: `/v3/projects/497d21b0cf2c46e9aacf7fed7ec1d787/users/c3e1a21999d749a58a1994ffcad508c5/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.206075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=497d21b0cf2c46e9aacf7fed7ec1d787, user_id=c3e1a21999d749a58a1994ffcad508c5, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:17.208839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:17.290015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1231/2444] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:17 2022] PUT /identity/v3/projects/497d21b0cf2c46e9aacf7fed7ec1d787/users/c3e1a21999d749a58a1994ffcad508c5/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 116 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:17.294137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:17.304105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a138ee-dbc1-48d7-8786-a7fd72c2a0d1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:17.356742 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=4,keystone:DELETE=1,keystone:SELECT=95 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:17.362474 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e6ab175-2a61-4ab6-a898-621d6f457a93 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:17.363255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab175-2a61-4ab6-a898-621d6f457a93 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.363542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab175-2a61-4ab6-a898-621d6f457a93 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.363818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e6ab175-2a61-4ab6-a898-621d6f457a93 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.364788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab175-2a61-4ab6-a898-621d6f457a93 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:17.366147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e6ab175-2a61-4ab6-a898-621d6f457a93 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:17.376396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1214/2445] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:17 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:17.380620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:17.386844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:17.392268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1781e079-0015-4aa0-ab8d-b4a4e9c3d4ce admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:17.396913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:17.397844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.398131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.398410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] PATH_INFO: `/v3/projects/497d21b0cf2c46e9aacf7fed7ec1d787/users/c3e1a21999d749a58a1994ffcad508c5/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.400417 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] RBAC: Authorizing `identity:create_grant(project_id=497d21b0cf2c46e9aacf7fed7ec1d787, user_id=c3e1a21999d749a58a1994ffcad508c5, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:17.403267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:17.413289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1232/2446] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:17 2022] PUT /identity/v3/projects/497d21b0cf2c46e9aacf7fed7ec1d787/users/c3e1a21999d749a58a1994ffcad508c5/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:17.425095 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.425386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.425665 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.452774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] MFA Rules not processed for user `c3e1a21999d749a58a1994ffcad508c5`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:17.454028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:17.520304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1215/2447] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:17 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:02:17.536772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:17.538501 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:17.595482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fca46843-27a8-4c59-8ba9-880b7b5c304b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:17.599457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:17.602781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:17.602781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:17.602781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:17.610260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:17.611689 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:17.621737 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1233/2448] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:17 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:18.642733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:18.652531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65751824-66bb-412d-95f6-1e04ed1e584f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:18.657240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e1a21999d749a58a1994ffcad508c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '497d21b0cf2c46e9aacf7fed7ec1d787', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:18.657903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:18.661806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:18.661806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:18.661806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:18.661806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:18.661806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:18.672779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1216/2449] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:02:18.676702 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:18.686050 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a621720c-0683-43a5-8e3b-70b3a052319b service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:18.698401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e1a21999d749a58a1994ffcad508c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '497d21b0cf2c46e9aacf7fed7ec1d787', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:18.698401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:18.698401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:18.698711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:18.698711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:18.698711 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:18.812945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1234/2450] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:18 2022] POST /identity/v3/projects => generated 355 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:18.817290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:18.828119 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b57bf0-35d8-4167-9077-e8a3bb13f01c tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:18.884488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e1a21999d749a58a1994ffcad508c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '497d21b0cf2c46e9aacf7fed7ec1d787', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:18.885152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:18.886079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:18.886363 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:18.887083 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:18.887420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:18.889040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:18.900387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1217/2451] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 84 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:02:18.936339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:18.946381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-454d72cc-ee22-48db-9101-5f5deff1bbd4 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:18.951429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'c3e1a21999d749a58a1994ffcad508c5', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '497d21b0cf2c46e9aacf7fed7ec1d787', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:18.952333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:18.952608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:18.952889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] PATH_INFO: `/v3/projects/5c079d5f4c85439183bc99d94f68af95` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:18.954057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorizing `identity:delete_project(project_id=5c079d5f4c85439183bc99d94f68af95)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:18.955990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:19.069100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5c079d5f4c85439183bc99d94f68af95', 'request_id': 'req-8abee52e-581f-4e1a-aa15-d44d6477db9e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:19.076799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5c079d5f4c85439183bc99d94f68af95', 'request_id': 'req-8abee52e-581f-4e1a-aa15-d44d6477db9e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:19.082934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5c079d5f4c85439183bc99d94f68af95', 'request_id': 'req-8abee52e-581f-4e1a-aa15-d44d6477db9e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:19.170559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] The token cache is being invalidate because project 5c079d5f4c85439183bc99d94f68af95 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:19.170856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:19.171497 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8abee52e-581f-4e1a-aa15-d44d6477db9e tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:19.173121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1235/2452] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:18 2022] DELETE /identity/v3/projects/5c079d5f4c85439183bc99d94f68af95 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:02:19.666193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:19.668633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:19.736955 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e6088a8-cd05-46ee-a589-55ef8a1eda65 tempest-VolumeQuotaClassesTest-1414322510 tempest-VolumeQuotaClassesTest-1414322510-project-admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:19.741328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:19.791631 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:19.793126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:19.793410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:19.793684 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] PATH_INFO: `/v3/users/dc0e0054c0a144eea24a3b80104718d0` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:19.794868 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] RBAC: Authorizing `identity:delete_user(user_id=dc0e0054c0a144eea24a3b80104718d0)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:19.796822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:19.983724 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dc0e0054c0a144eea24a3b80104718d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:19.984438 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dc0e0054c0a144eea24a3b80104718d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.078907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dc0e0054c0a144eea24a3b80104718d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.086207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dc0e0054c0a144eea24a3b80104718d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.098765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dc0e0054c0a144eea24a3b80104718d0'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.104768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1218/2453] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:19 2022] DELETE /identity/v3/users/dc0e0054c0a144eea24a3b80104718d0 => generated 0 bytes in 439 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:20.109358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:20.116169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:20.116169 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:20.129895 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:20.131098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:20.178968 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:20.190833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1236/2454] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:20.200996 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:20.202822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:20.264439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc1d909e-4461-4c7a-84c0-d12b806d3e99 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:20.268100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:20.268738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:20.269024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:20.269302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:20.278999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:20.283147 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:20.290746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1219/2455] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:02:20.738451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:20.740064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:20.753582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a5a9932-6169-4567-b843-9ed52a834a64 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:20.758651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:20.759631 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:20.760060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:20.760474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] PATH_INFO: `/v3/projects/11f37382fe3c46c988b473d6473949e3` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:20.761812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] RBAC: Authorizing `identity:delete_project(project_id=11f37382fe3c46c988b473d6473949e3)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:20.763808 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:20.879914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '11f37382fe3c46c988b473d6473949e3', 'request_id': 'req-00396de7-a91a-4c9b-932d-939bcf306b69'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.888757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '11f37382fe3c46c988b473d6473949e3', 'request_id': 'req-00396de7-a91a-4c9b-932d-939bcf306b69'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.894930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '11f37382fe3c46c988b473d6473949e3', 'request_id': 'req-00396de7-a91a-4c9b-932d-939bcf306b69'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.998879 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] The token cache is being invalidate because project 11f37382fe3c46c988b473d6473949e3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:20.999177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:20.999803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.001368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1237/2456] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:20 2022] DELETE /identity/v3/projects/11f37382fe3c46c988b473d6473949e3 => generated 0 bytes in 263 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:21.005374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:21.015198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030eba4c-5dfe-44fe-8462-d809786ddb10 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:21.040922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:21.070637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:21.071508 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:21.071792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:21.072163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] PATH_INFO: `/v3/users/c3e1a21999d749a58a1994ffcad508c5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:21.073365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c3e1a21999d749a58a1994ffcad508c5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:21.075289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:21.263276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c3e1a21999d749a58a1994ffcad508c5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.263963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c3e1a21999d749a58a1994ffcad508c5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.352330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c3e1a21999d749a58a1994ffcad508c5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.359369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c3e1a21999d749a58a1994ffcad508c5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.371179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-060381d0-c913-436c-84c2-da7c69ea8741 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c3e1a21999d749a58a1994ffcad508c5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.377106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1220/2457] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:21 2022] DELETE /identity/v3/users/c3e1a21999d749a58a1994ffcad508c5 => generated 0 bytes in 372 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:21.778334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:21.779806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:21.793528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00396de7-a91a-4c9b-932d-939bcf306b69 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:21.851412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:21.852419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:21.852709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:21.852986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] PATH_INFO: `/v3/projects/497d21b0cf2c46e9aacf7fed7ec1d787` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:21.854174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=497d21b0cf2c46e9aacf7fed7ec1d787)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:21.856239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:21.942079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '497d21b0cf2c46e9aacf7fed7ec1d787', 'request_id': 'req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.948191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '497d21b0cf2c46e9aacf7fed7ec1d787', 'request_id': 'req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:21.954300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '497d21b0cf2c46e9aacf7fed7ec1d787', 'request_id': 'req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:22.060200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] The token cache is being invalidate because project 497d21b0cf2c46e9aacf7fed7ec1d787 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:22.060492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:22.061110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-050cdc43-56eb-4f8e-bfa5-6895d069bb7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:22.062687 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1238/2458] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:21 2022] DELETE /identity/v3/projects/497d21b0cf2c46e9aacf7fed7ec1d787 => generated 0 bytes in 285 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:22.067625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.068123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.106258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.106258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:22.106258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:22.147784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1221/2459] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:22.155668 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.155995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.156310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.169439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:22.175423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:22.176684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:22.185191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1239/2460] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:02:22.189277 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.199506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1a0a623-7750-4ecf-b7f0-e342f0861d9b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.204204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.204865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.205149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.205423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.206145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:22.206450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.208069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:22.223252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1222/2461] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:02:22.225060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.236140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b5c42ee-a304-46d0-886c-5ce046427f6d None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.241257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.244028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.244028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.244028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.244028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.245146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:22.361633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1240/2462] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:22 2022] POST /identity/v3/projects => generated 383 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:22.365965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.376466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c13a543-dd76-4f6b-819a-fad3b1164436 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.407632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:22.433228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.434607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.434898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.435180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.435915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.437849 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:22.517770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1223/2463] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:22 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:22.519196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.530111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6965ce5f-806b-4c5d-aa9d-570805a0910c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.534725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.535384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.535677 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.536229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.537092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.538739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:22.549304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1241/2464] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:22.553100 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.562959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db31c4c-06d5-46cf-ae42-5dfef8738fb3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.568165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6af555b-614d-4781-91c0-d28f06f0e10e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.568821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af555b-614d-4781-91c0-d28f06f0e10e admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.569104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af555b-614d-4781-91c0-d28f06f0e10e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.569423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6af555b-614d-4781-91c0-d28f06f0e10e admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.570150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af555b-614d-4781-91c0-d28f06f0e10e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.571729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6af555b-614d-4781-91c0-d28f06f0e10e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:22.581619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1224/2465] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:22 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:02:22.588803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.599593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d13567b9-7753-4f0b-a97e-12ceda123504 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.604357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.605882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.606191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.606473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] PATH_INFO: `/v3/projects/e22bb0b0cd694a8480c5b1cb44d82658/users/d747824aaad946d3883b9d2877122f1c/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.621434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e22bb0b0cd694a8480c5b1cb44d82658, user_id=d747824aaad946d3883b9d2877122f1c, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.624194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:22.771733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1242/2466] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:22 2022] PUT /identity/v3/projects/e22bb0b0cd694a8480c5b1cb44d82658/users/d747824aaad946d3883b9d2877122f1c/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:02:22.782731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.783030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.783323 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.812482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] MFA Rules not processed for user `d747824aaad946d3883b9d2877122f1c`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:22.813892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:22.880977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1225/2467] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:22 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:22.885359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:22.947729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94662aa2-967c-4cc4-81a0-68103b150ec4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:22.953163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:22.953871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:22.954179 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:22.954454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:22.955176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:22.957142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.040621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1243/2468] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:22 2022] POST /identity/v3/projects => generated 383 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:02:23.044653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.055566 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff325be-df31-4dd1-a29f-39578dddba14 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.111327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.112045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.112336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.112622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.113326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.115261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.176483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1226/2469] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:23 2022] POST /identity/v3/users => generated 298 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:23.180438 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.190828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49ac8e1f-c881-4758-bd9c-077513901cc4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.195571 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.196263 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.196553 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.196835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.197598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.199784 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.210751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1244/2470] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:23.215515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.225804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e476f25b-4a60-468d-a371-38ad264516b7 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.230671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.231556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.231847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.232158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] PATH_INFO: `/v3/projects/4e9f7499394c4e81b9c35e7b4a3c34d2/users/5e08bc759e004b76ae1b96dfc6fc9df3/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.247766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e9f7499394c4e81b9c35e7b4a3c34d2, user_id=5e08bc759e004b76ae1b96dfc6fc9df3, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.250510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.352326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1227/2471] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:23 2022] PUT /identity/v3/projects/4e9f7499394c4e81b9c35e7b4a3c34d2/users/5e08bc759e004b76ae1b96dfc6fc9df3/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 138 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:23.356272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.366215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a235635-ac92-4be7-b5e5-7998cc4bb514 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.421649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15575fd4-8704-49d5-aca2-7c2c22bf5179 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.422346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575fd4-8704-49d5-aca2-7c2c22bf5179 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.422632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575fd4-8704-49d5-aca2-7c2c22bf5179 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.422910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15575fd4-8704-49d5-aca2-7c2c22bf5179 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.423632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575fd4-8704-49d5-aca2-7c2c22bf5179 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.425255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15575fd4-8704-49d5-aca2-7c2c22bf5179 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.435413 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1245/2472] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:23 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:23.439526 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.450477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38c9bf54-2bf6-4fa8-b5c7-9fd19ee11202 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.455089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.455998 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.456288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.456568 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] PATH_INFO: `/v3/projects/4e9f7499394c4e81b9c35e7b4a3c34d2/users/5e08bc759e004b76ae1b96dfc6fc9df3/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.458544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e9f7499394c4e81b9c35e7b4a3c34d2, user_id=5e08bc759e004b76ae1b96dfc6fc9df3, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.461248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.464301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:23.472097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1228/2473] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:23 2022] PUT /identity/v3/projects/4e9f7499394c4e81b9c35e7b4a3c34d2/users/5e08bc759e004b76ae1b96dfc6fc9df3/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:23.489971 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.490265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.490605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.498066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:23.519214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] MFA Rules not processed for user `5e08bc759e004b76ae1b96dfc6fc9df3`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:23.520493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:23.590886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1246/2474] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:23 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:23.602178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.604025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:23.661373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44277253-4df6-432f-a2eb-1e030d8ccc13 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.665274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.665933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.666218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.666624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.682878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.685001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.696188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1229/2475] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:23.793833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:23.803961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d1a591-392c-490e-ad27-ac26f088d6a4 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:23.807649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:23.808986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:23.809278 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:23.809564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:23.876347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:23.878031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:23.889292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1247/2476] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:23 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:24.187852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:24.189327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:24.254274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10913eb-6368-493e-acd7-c88e7b24694b service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:24.258784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:24.259654 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:24.259968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:24.260245 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] PATH_INFO: `/v3/users/5e69fc12fcc34fa39f67dd156cd860a4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:24.261509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e69fc12fcc34fa39f67dd156cd860a4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:24.263420 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:24.445760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e69fc12fcc34fa39f67dd156cd860a4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.446433 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e69fc12fcc34fa39f67dd156cd860a4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.514001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e69fc12fcc34fa39f67dd156cd860a4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.521506 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e69fc12fcc34fa39f67dd156cd860a4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.534524 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-210528dc-a69d-4503-9e80-857960e13579 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e69fc12fcc34fa39f67dd156cd860a4'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.543084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1230/2477] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:24 2022] DELETE /identity/v3/users/5e69fc12fcc34fa39f67dd156cd860a4 => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:24.771388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:24.782894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eeb7630c-7f40-4ca0-aa18-bbf29526c434 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:24.840272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:24.841135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:24.841416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:24.841772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] PATH_INFO: `/v3/projects/82c80e6e109741d1bd83c7b81368a87f` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:24.842931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] RBAC: Authorizing `identity:delete_project(project_id=82c80e6e109741d1bd83c7b81368a87f)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:24.844864 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:24.929298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:DELETE=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:24.942486 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '82c80e6e109741d1bd83c7b81368a87f', 'request_id': 'req-c66e3ab3-78da-4f58-9a3e-5aede52296ba'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.948598 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '82c80e6e109741d1bd83c7b81368a87f', 'request_id': 'req-c66e3ab3-78da-4f58-9a3e-5aede52296ba'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:24.954586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '82c80e6e109741d1bd83c7b81368a87f', 'request_id': 'req-c66e3ab3-78da-4f58-9a3e-5aede52296ba'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:25.021565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] The token cache is being invalidate because project 82c80e6e109741d1bd83c7b81368a87f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:25.021861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:25.022508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:25.024336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1248/2478] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:24 2022] DELETE /identity/v3/projects/82c80e6e109741d1bd83c7b81368a87f => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:26.229463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:26.230339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:26.230637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:26.248416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:26.249622 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:26.263598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:26.306174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1231/2479] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 12:02:26 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:26.436958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:26.441799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:26.508479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c66e3ab3-78da-4f58-9a3e-5aede52296ba admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:26.514204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ed093b1-9cc5-4f8e-867c-8de345dec76d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:26.514852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed093b1-9cc5-4f8e-867c-8de345dec76d service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:26.515139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed093b1-9cc5-4f8e-867c-8de345dec76d service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:26.515420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ed093b1-9cc5-4f8e-867c-8de345dec76d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:26.523404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed093b1-9cc5-4f8e-867c-8de345dec76d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:26.525017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ed093b1-9cc5-4f8e-867c-8de345dec76d service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:26.533303 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1249/2480] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 12:02:26 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 12:02:27.046676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:27.049168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:27.137537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20683b44-6f93-4b82-9511-ad447a15ad2b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:27.142556 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:27.143729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:27.144156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:27.144538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] PATH_INFO: `/v3/users/d747824aaad946d3883b9d2877122f1c` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:27.145871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] RBAC: Authorizing `identity:delete_user(user_id=d747824aaad946d3883b9d2877122f1c)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:27.148902 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:27.367362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd747824aaad946d3883b9d2877122f1c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:27.368805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd747824aaad946d3883b9d2877122f1c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:27.444837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd747824aaad946d3883b9d2877122f1c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:27.456069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd747824aaad946d3883b9d2877122f1c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:27.472395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd747824aaad946d3883b9d2877122f1c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:27.484126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1232/2481] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:27 2022] DELETE /identity/v3/users/d747824aaad946d3883b9d2877122f1c => generated 0 bytes in 438 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:27.489714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:27.490490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:27.490985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:27.515854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:27.517209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:27.606018 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1250/2482] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:27.618513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:27.620029 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:27.670949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:27.682853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ce90ee4-bb64-48bc-9962-8034897ffb6f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:27.692530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:27.693196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:27.693480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:27.693767 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:27.705778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:27.707305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:27.718272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1233/2483] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:28.200057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:28.200419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:28.228388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab2551dc-83d6-4047-a21f-0b2776f6026a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:28.237577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:28.239526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:28.239823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:28.240690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] PATH_INFO: `/v3/projects/e22bb0b0cd694a8480c5b1cb44d82658` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:28.242564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e22bb0b0cd694a8480c5b1cb44d82658)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:28.246479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:28.392482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:28.399944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e22bb0b0cd694a8480c5b1cb44d82658', 'request_id': 'req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.414082 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e22bb0b0cd694a8480c5b1cb44d82658', 'request_id': 'req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.426505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e22bb0b0cd694a8480c5b1cb44d82658', 'request_id': 'req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.517349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] The token cache is being invalidate because project e22bb0b0cd694a8480c5b1cb44d82658 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:28.518012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.518327 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.522873 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1251/2484] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:28 2022] DELETE /identity/v3/projects/e22bb0b0cd694a8480c5b1cb44d82658 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:28.524112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:28.541593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eba8f-2350-4bd0-929a-cd8daabd39d0 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:28.621284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:28.622301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:28.622747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:28.623118 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] PATH_INFO: `/v3/users/5e08bc759e004b76ae1b96dfc6fc9df3` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:28.626675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5e08bc759e004b76ae1b96dfc6fc9df3)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:28.629905 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:28.848108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5e08bc759e004b76ae1b96dfc6fc9df3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.852492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5e08bc759e004b76ae1b96dfc6fc9df3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.913800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5e08bc759e004b76ae1b96dfc6fc9df3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.923326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5e08bc759e004b76ae1b96dfc6fc9df3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.946056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9595799b-c2e4-4c99-9654-6ec29c87a3d1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5e08bc759e004b76ae1b96dfc6fc9df3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:28.955682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1234/2485] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:28 2022] DELETE /identity/v3/users/5e08bc759e004b76ae1b96dfc6fc9df3 => generated 0 bytes in 432 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:29.687620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:29.697882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:29.703537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68b9261-68f5-4bf9-9f30-ca81ac77dc15 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:29.759799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:29.760691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:29.760978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:29.761754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] PATH_INFO: `/v3/projects/4e9f7499394c4e81b9c35e7b4a3c34d2` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:29.762931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e9f7499394c4e81b9c35e7b4a3c34d2)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:29.764889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:29.920785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e9f7499394c4e81b9c35e7b4a3c34d2', 'request_id': 'req-aa1a50f6-6405-4add-a7b2-6f26f4e11585'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:29.928666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e9f7499394c4e81b9c35e7b4a3c34d2', 'request_id': 'req-aa1a50f6-6405-4add-a7b2-6f26f4e11585'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:29.940321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e9f7499394c4e81b9c35e7b4a3c34d2', 'request_id': 'req-aa1a50f6-6405-4add-a7b2-6f26f4e11585'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:30.062721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] The token cache is being invalidate because project 4e9f7499394c4e81b9c35e7b4a3c34d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:30.069656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:30.069656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-aa1a50f6-6405-4add-a7b2-6f26f4e11585 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:30.069656 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1252/2486] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:29 2022] DELETE /identity/v3/projects/4e9f7499394c4e81b9c35e7b4a3c34d2 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:30.071587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.072460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.075908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.104181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:30.105608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:30.177411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1235/2487] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:30.187104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.187403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.187611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.214090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:30.215492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:30.224432 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1253/2488] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:30.228362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:30.238514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:30.242536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8443e6d-b427-4a14-acd3-3e3064b7d287 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:30.248343 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:30.249149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.249529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.250042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.252178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:30.254264 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:30.255179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:30.271945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1236/2489] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:02:30.275462 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:30.286720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba1d0649-a3bc-440c-bb39-33bb7923e35c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:30.291955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:30.293534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.293534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.293534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.293969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:30.295909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:30.388161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1254/2490] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:30 2022] POST /identity/v3/projects => generated 389 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:30.389589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:30.400322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2c399b8-5cca-478f-9131-0d993476c02d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:30.466626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:30.467377 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.467661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.467967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.468671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:30.470914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:30.572816 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1237/2491] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:30 2022] POST /identity/v3/users => generated 295 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:30.577378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:30.588505 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfbee4d9-cded-434b-b074-0d262548fe1f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:30.594244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:30.602163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.602463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.602747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.603504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:30.606569 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:30.621325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1255/2492] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:30.624778 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:30.637446 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a43a3a9-174f-4525-8793-01e4e5fef178 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:30.643107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8de0a572-fd59-41e6-a2d9-3201c544b121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:30.643784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a572-fd59-41e6-a2d9-3201c544b121 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.644103 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a572-fd59-41e6-a2d9-3201c544b121 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.644385 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8de0a572-fd59-41e6-a2d9-3201c544b121 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.645117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0a572-fd59-41e6-a2d9-3201c544b121 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:30.646729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8de0a572-fd59-41e6-a2d9-3201c544b121 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:30.658068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1238/2493] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:30 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:02:30.662203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:30.679096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e04466-eb3d-45f8-8c1d-aa260863e4be admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:30.685001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:30.689010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.689010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.689010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] PATH_INFO: `/v3/projects/80d134b4c0704122b581d627292cf9d5/users/a2d99b33a459452d9b1cade2c581fd6c/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.702951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] RBAC: Authorizing `identity:create_grant(project_id=80d134b4c0704122b581d627292cf9d5, user_id=a2d99b33a459452d9b1cade2c581fd6c, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:30.706078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:30.783746 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:30.792871 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1256/2494] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:30 2022] PUT /identity/v3/projects/80d134b4c0704122b581d627292cf9d5/users/a2d99b33a459452d9b1cade2c581fd6c/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 131 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:30.804674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:30.805186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:30.805709 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:30.863089 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] MFA Rules not processed for user `a2d99b33a459452d9b1cade2c581fd6c`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:30.864964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:30.969414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1239/2495] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:30 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:30.973445 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.083706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-285d94e2-0279-410e-a0cb-1e7788fe36c7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.089020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.090699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.091116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.091461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.092281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.094342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.209776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1257/2496] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:30 2022] POST /identity/v3/projects => generated 387 bytes in 237 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:02:31.210747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.226768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c521643d-10b9-41cd-bde6-8acbfd126e1b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.296376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.297010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.297299 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.297583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.298286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.300273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.310639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:31.391266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.391911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1240/2497] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:31 2022] POST /identity/v3/users => generated 300 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:02:31.403761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-361b42f7-c789-4301-b780-3fd0e4d850e9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.408847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.409754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.410112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.410610 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.411430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.413166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.426604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1258/2498] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:31.431068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.447930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0462b18-662b-452b-9a20-384c31a6aec2 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.454971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.454971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.455216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.456903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] PATH_INFO: `/v3/projects/93fb073cf2d940569479ec11b607ee17/users/447ac03e9b394ce5af14bfcc4d280d1c/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.478670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93fb073cf2d940569479ec11b607ee17, user_id=447ac03e9b394ce5af14bfcc4d280d1c, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.484256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.560444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1241/2499] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:31 2022] PUT /identity/v3/projects/93fb073cf2d940569479ec11b607ee17/users/447ac03e9b394ce5af14bfcc4d280d1c/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:02:31.565775 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.578154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24693fd4-4e44-42e8-aaad-1ca93d0b95f4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.658374 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1830ae17-1859-4dc3-9016-f0c5cc7e6c23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.659061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ae17-1859-4dc3-9016-f0c5cc7e6c23 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.659353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ae17-1859-4dc3-9016-f0c5cc7e6c23 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.659644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1830ae17-1859-4dc3-9016-f0c5cc7e6c23 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.660495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830ae17-1859-4dc3-9016-f0c5cc7e6c23 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.664651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1830ae17-1859-4dc3-9016-f0c5cc7e6c23 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.678751 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1259/2500] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:31 2022] GET /identity/v3/roles => generated 1420 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:31.682421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.702486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a86ddd93-a7cb-4420-aace-e874f699f9a4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.702486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.702486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.702486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.703473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] PATH_INFO: `/v3/projects/93fb073cf2d940569479ec11b607ee17/users/447ac03e9b394ce5af14bfcc4d280d1c/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.704428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] RBAC: Authorizing `identity:create_grant(project_id=93fb073cf2d940569479ec11b607ee17, user_id=447ac03e9b394ce5af14bfcc4d280d1c, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.707154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.720371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1242/2501] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:31 2022] PUT /identity/v3/projects/93fb073cf2d940569479ec11b607ee17/users/447ac03e9b394ce5af14bfcc4d280d1c/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:31.731349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401fba-35c6-4271-a876-3253703aa40b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.731955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401fba-35c6-4271-a876-3253703aa40b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.732173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56401fba-35c6-4271-a876-3253703aa40b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.763181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-56401fba-35c6-4271-a876-3253703aa40b None None] MFA Rules not processed for user `447ac03e9b394ce5af14bfcc4d280d1c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:31.764517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-56401fba-35c6-4271-a876-3253703aa40b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:31.838164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:31.849676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1260/2502] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:31 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:02:31.860422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:31.862432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:31.939294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04a8431c-799f-4391-97bc-7ff182b595e2 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:31.944465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73d0a7a-6c28-41c0-a0d7-c5e91a1fab10 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:31.945884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d0a7a-6c28-41c0-a0d7-c5e91a1fab10 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:31.946419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d0a7a-6c28-41c0-a0d7-c5e91a1fab10 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:31.946908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73d0a7a-6c28-41c0-a0d7-c5e91a1fab10 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:31.959925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d0a7a-6c28-41c0-a0d7-c5e91a1fab10 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:31.961440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73d0a7a-6c28-41c0-a0d7-c5e91a1fab10 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:31.974851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1243/2503] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:31 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:32.134037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56401fba-35c6-4271-a876-3253703aa40b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:32.208581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56401fba-35c6-4271-a876-3253703aa40b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:32.213579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:32.214735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:32.214985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:32.215480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] PATH_INFO: `/v3/users/a2d99b33a459452d9b1cade2c581fd6c` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:32.217295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] RBAC: Authorizing `identity:delete_user(user_id=a2d99b33a459452d9b1cade2c581fd6c)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:32.219530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:32.392360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'a2d99b33a459452d9b1cade2c581fd6c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:32.393298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'a2d99b33a459452d9b1cade2c581fd6c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:32.473785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'a2d99b33a459452d9b1cade2c581fd6c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:32.482032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'a2d99b33a459452d9b1cade2c581fd6c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:32.494664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'a2d99b33a459452d9b1cade2c581fd6c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:32.501806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1261/2504] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:32 2022] DELETE /identity/v3/users/a2d99b33a459452d9b1cade2c581fd6c => generated 0 bytes in 370 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:32.506469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:32.506982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:32.507105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:32.526483 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:32.527912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:32.593725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1244/2505] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:32.604300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:32.606526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:32.671258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735980d-b1d6-46ec-8911-90a03c6eea67 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:32.680065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:32.680753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:32.681039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:32.681322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:32.697525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:32.699077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:32.710063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1262/2506] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:33.302025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:33.303761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:33.333471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24ef0ee2-6524-4c33-a0bd-968431803aa0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:33.342598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:33.343496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:33.343780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:33.344085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] PATH_INFO: `/v3/projects/80d134b4c0704122b581d627292cf9d5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:33.345293 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] RBAC: Authorizing `identity:delete_project(project_id=80d134b4c0704122b581d627292cf9d5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:33.347671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:33.455986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '80d134b4c0704122b581d627292cf9d5', 'request_id': 'req-368433ad-2ccd-4609-afd7-b289b8bd9723'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.457388 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=5,keystone:SELECT=91,keystone:DELETE=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:33.468258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '80d134b4c0704122b581d627292cf9d5', 'request_id': 'req-368433ad-2ccd-4609-afd7-b289b8bd9723'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.531352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '80d134b4c0704122b581d627292cf9d5', 'request_id': 'req-368433ad-2ccd-4609-afd7-b289b8bd9723'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.559862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] The token cache is being invalidate because project 80d134b4c0704122b581d627292cf9d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:33.560380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.561061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.562672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1245/2507] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:33 2022] DELETE /identity/v3/projects/80d134b4c0704122b581d627292cf9d5 => generated 0 bytes in 261 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:33.568121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:33.582982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf92012-c43a-42cc-9bb6-d62826c7a178 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:33.596427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:33.673563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:33.674533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:33.675020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:33.675131 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] PATH_INFO: `/v3/users/447ac03e9b394ce5af14bfcc4d280d1c` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:33.684504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=447ac03e9b394ce5af14bfcc4d280d1c)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:33.684504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:33.890034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '447ac03e9b394ce5af14bfcc4d280d1c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.890638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '447ac03e9b394ce5af14bfcc4d280d1c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.976447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '447ac03e9b394ce5af14bfcc4d280d1c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:33.988815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '447ac03e9b394ce5af14bfcc4d280d1c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.004074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-758af84b-8682-49ab-b86d-b2a0d4a975e6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '447ac03e9b394ce5af14bfcc4d280d1c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.011806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1263/2508] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:33 2022] DELETE /identity/v3/users/447ac03e9b394ce5af14bfcc4d280d1c => generated 0 bytes in 444 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:34.424811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:34.426489 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:34.448443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-368433ad-2ccd-4609-afd7-b289b8bd9723 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:34.520966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:34.523194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:34.523703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:34.524071 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] PATH_INFO: `/v3/projects/93fb073cf2d940569479ec11b607ee17` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:34.525422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] RBAC: Authorizing `identity:delete_project(project_id=93fb073cf2d940569479ec11b607ee17)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:34.527455 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:34.675945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '93fb073cf2d940569479ec11b607ee17', 'request_id': 'req-1deb2267-842c-4881-bb9f-c92e4a029490'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.696153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '93fb073cf2d940569479ec11b607ee17', 'request_id': 'req-1deb2267-842c-4881-bb9f-c92e4a029490'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.776824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '93fb073cf2d940569479ec11b607ee17', 'request_id': 'req-1deb2267-842c-4881-bb9f-c92e4a029490'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.796966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] The token cache is being invalidate because project 93fb073cf2d940569479ec11b607ee17 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:34.797265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.797951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1deb2267-842c-4881-bb9f-c92e4a029490 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:34.800085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1246/2509] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:34 2022] DELETE /identity/v3/projects/93fb073cf2d940569479ec11b607ee17 => generated 0 bytes in 376 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:34.805229 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:34.805757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:34.806027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:34.833597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:34.834919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:34.927320 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1264/2510] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:34.935412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:34.935705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:34.936047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:34.961235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:34.962473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:34.975043 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1247/2511] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:34 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 41 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:34.976986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:34.991409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc2b800c-2f91-41f9-8b4e-73537d9cc2f0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:34.997368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:34.998233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:34.998660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:34.999107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.000212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:35.000594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.002525 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.015905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1265/2512] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:02:35.020335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.031641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8be8dc61-a052-4d59-9044-ebe200a0594d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.036567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.037230 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.037538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.037793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.038520 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.040521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.134129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1248/2513] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:35 2022] POST /identity/v3/projects => generated 367 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:02:35.135724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.148822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b58cc68-b02e-430e-84f1-e1dab2357434 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.180899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:35.225270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.226118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.226510 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.227001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.227850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.230127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.358648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1266/2514] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:35 2022] POST /identity/v3/users => generated 284 bytes in 223 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:35.361108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.372974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99bc4e01-8c29-4e89-8585-5977e82a547c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.378059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.378864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.379354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.379687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.380595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.382329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.394496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1249/2515] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:35.398440 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.411975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f63d94c-0e2e-4290-a18f-60226fd5db1a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.418378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-013df2b8-0cb7-47d4-845e-31a13e4cd5d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.419056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013df2b8-0cb7-47d4-845e-31a13e4cd5d6 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.419346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013df2b8-0cb7-47d4-845e-31a13e4cd5d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.419637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-013df2b8-0cb7-47d4-845e-31a13e4cd5d6 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.422212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013df2b8-0cb7-47d4-845e-31a13e4cd5d6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.422420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-013df2b8-0cb7-47d4-845e-31a13e4cd5d6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.434740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1267/2516] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:35 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:02:35.438630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.450325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c440fc6f-cd26-437e-bf4d-79eed3266ed1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.455320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.456361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.456711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.457072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] PATH_INFO: `/v3/projects/6bd0b7ca20504630bf714b3a7cebf107/users/23ea6276221b401184b54538305f6582/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.473875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6bd0b7ca20504630bf714b3a7cebf107, user_id=23ea6276221b401184b54538305f6582, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.477194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.560093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1250/2517] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:35 2022] PUT /identity/v3/projects/6bd0b7ca20504630bf714b3a7cebf107/users/23ea6276221b401184b54538305f6582/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:35.561463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.561756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.562083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.593657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] MFA Rules not processed for user `23ea6276221b401184b54538305f6582`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:35.594110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:35.673116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1268/2518] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:35 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:35.675494 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.681466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=10,keystone:SELECT=86,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:35.746750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a53d67f5-3094-4912-b6d1-be003be22d17 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.751689 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.752436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.753171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.753461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.754559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.757346 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:35.858266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1251/2519] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:35 2022] POST /identity/v3/projects => generated 367 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:35.862567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:35.874264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdc938cd-ea5b-4ef0-b8e6-12483b7fd320 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:35.949806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:35.950813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:35.951180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:35.951530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:35.952536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:35.955558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:36.235608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1269/2520] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:35 2022] POST /identity/v3/users => generated 290 bytes in 374 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:36.242939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:36.253863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20c8eefb-c66c-4839-9f33-e664cbeccd4f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:36.258514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:36.259564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:36.259859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:36.260170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:36.260986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:36.263158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:36.274736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1252/2521] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:36.278647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:36.292206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57f24f42-8c16-42ec-9607-cd14c04d3e51 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:36.297076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:36.300786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:36.300786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:36.300786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] PATH_INFO: `/v3/projects/97e42539fb1c4a089ebe8429c2f35f7c/users/48bb562e767641f99117e2d20fae5727/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:36.319224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e42539fb1c4a089ebe8429c2f35f7c, user_id=48bb562e767641f99117e2d20fae5727, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:36.322012 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:36.326710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:36.434582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1270/2522] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:36 2022] PUT /identity/v3/projects/97e42539fb1c4a089ebe8429c2f35f7c/users/48bb562e767641f99117e2d20fae5727/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:02:36.438838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:36.454918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22bd7d5f-94dd-4d0b-9774-e3fa3f0d5892 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:36.531087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-690b898b-cc0e-4d9c-8cfe-7834ab586dd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:36.532306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b898b-cc0e-4d9c-8cfe-7834ab586dd7 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:36.532765 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b898b-cc0e-4d9c-8cfe-7834ab586dd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:36.533196 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-690b898b-cc0e-4d9c-8cfe-7834ab586dd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:36.534087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b898b-cc0e-4d9c-8cfe-7834ab586dd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:36.538199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-690b898b-cc0e-4d9c-8cfe-7834ab586dd7 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:36.547907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1253/2523] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:36 2022] GET /identity/v3/roles => generated 1420 bytes in 110 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:36.552492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:36.564708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc73bb00-f0e1-4cb3-8d34-5cc198b97b8b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:36.569837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:36.570740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:36.571032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:36.573802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] PATH_INFO: `/v3/projects/97e42539fb1c4a089ebe8429c2f35f7c/users/48bb562e767641f99117e2d20fae5727/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:36.574352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] RBAC: Authorizing `identity:create_grant(project_id=97e42539fb1c4a089ebe8429c2f35f7c, user_id=48bb562e767641f99117e2d20fae5727, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:36.579114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:36.591222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1271/2524] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:36 2022] PUT /identity/v3/projects/97e42539fb1c4a089ebe8429c2f35f7c/users/48bb562e767641f99117e2d20fae5727/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 40 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:36.602445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:36.604486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:36.604486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:36.637175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] MFA Rules not processed for user `48bb562e767641f99117e2d20fae5727`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:36.658779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:36.727278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:36.731121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1254/2525] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:36 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:36.742938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:36.744613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:36.807732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03105d8e-aefd-45b8-9291-f45d5ca3358a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:36.814065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:36.814735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:36.815027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:36.815311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:36.826266 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:36.827750 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:36.838886 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1272/2526] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:36 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:37.144872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:37.154876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-965b4f61-d2b2-4ba9-944c-b77a155ec5d4 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:37.158964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55c9227d-d300-4c5a-b2f5-8cc111eab684 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:37.159698 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9227d-d300-4c5a-b2f5-8cc111eab684 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:37.160214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9227d-d300-4c5a-b2f5-8cc111eab684 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:37.160570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55c9227d-d300-4c5a-b2f5-8cc111eab684 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:37.225185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9227d-d300-4c5a-b2f5-8cc111eab684 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:37.226632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55c9227d-d300-4c5a-b2f5-8cc111eab684 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:37.237669 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1255/2527] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:37 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:43.916356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:43.978870 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26509db4-9460-4355-a0cd-22a25e154bd0 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:43.983923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:43.984801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:43.985118 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:43.985397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] PATH_INFO: `/v3/users/23ea6276221b401184b54538305f6582` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:43.986579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=23ea6276221b401184b54538305f6582)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:43.988533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:44.184485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '23ea6276221b401184b54538305f6582'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:44.185137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '23ea6276221b401184b54538305f6582'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:44.261887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '23ea6276221b401184b54538305f6582'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:44.269338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '23ea6276221b401184b54538305f6582'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:44.282347 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '23ea6276221b401184b54538305f6582'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:44.289478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1273/2528] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:43 2022] DELETE /identity/v3/users/23ea6276221b401184b54538305f6582 => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:44.295453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:44.295943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:44.296042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:44.316479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:44.317755 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:44.380499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1256/2529] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:44.390544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:44.392100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:44.404113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:44.456499 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b01128-7276-463b-9ed3-d415521bf5b2 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:44.460367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:44.461151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:44.461825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:44.462427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:44.473890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:44.475301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:44.485384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1274/2530] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:44 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:44.893797 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:44.895236 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:44.907962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2876cd04-7257-4acf-8225-1b7580bb3eed None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:44.913088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:44.913977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:44.914263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:44.914545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] PATH_INFO: `/v3/projects/6bd0b7ca20504630bf714b3a7cebf107` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:44.915906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6bd0b7ca20504630bf714b3a7cebf107)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:44.917990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:45.026932 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6bd0b7ca20504630bf714b3a7cebf107', 'request_id': 'req-4671cf5c-a37e-4a05-93a4-bff477036485'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.033882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6bd0b7ca20504630bf714b3a7cebf107', 'request_id': 'req-4671cf5c-a37e-4a05-93a4-bff477036485'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.125418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6bd0b7ca20504630bf714b3a7cebf107', 'request_id': 'req-4671cf5c-a37e-4a05-93a4-bff477036485'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.139411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] The token cache is being invalidate because project 6bd0b7ca20504630bf714b3a7cebf107 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:45.139710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.140833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-4671cf5c-a37e-4a05-93a4-bff477036485 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.142300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1257/2531] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:44 2022] DELETE /identity/v3/projects/6bd0b7ca20504630bf714b3a7cebf107 => generated 0 bytes in 249 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:45.146766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:45.156728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043e388b-5a2a-4006-83fc-38d8fbdf683f service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:45.214201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:45.215109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:45.215387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:45.215661 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] PATH_INFO: `/v3/users/48bb562e767641f99117e2d20fae5727` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:45.216881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] RBAC: Authorizing `identity:delete_user(user_id=48bb562e767641f99117e2d20fae5727)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:45.218826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:45.373931 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '48bb562e767641f99117e2d20fae5727'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.374533 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '48bb562e767641f99117e2d20fae5727'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.427537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '48bb562e767641f99117e2d20fae5727'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.436191 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '48bb562e767641f99117e2d20fae5727'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.448822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '48bb562e767641f99117e2d20fae5727'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:45.454945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1275/2532] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:45 2022] DELETE /identity/v3/users/48bb562e767641f99117e2d20fae5727 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:45.575182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:45.575675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:45.575906 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:45.594737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:45.596014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:45.614366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:45.653304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1258/2533] 10.222.0.40 () {58 vars in 1009 bytes} [Thu Jun 16 12:02:45 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:45.679226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:45.680772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:45.715289 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:45.756001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3882a319-9b7d-40e5-8870-4220c5ffb291 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:45.761203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3295b9d6-44b7-4873-9ffa-4ce997a8e9b1 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:45.762492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295b9d6-44b7-4873-9ffa-4ce997a8e9b1 service swift] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:45.762923 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295b9d6-44b7-4873-9ffa-4ce997a8e9b1 service swift] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:45.763369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3295b9d6-44b7-4873-9ffa-4ce997a8e9b1 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:45.773377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295b9d6-44b7-4873-9ffa-4ce997a8e9b1 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:45.775009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3295b9d6-44b7-4873-9ffa-4ce997a8e9b1 service swift] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:45.784861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1276/2534] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 12:02:45 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 106 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jun 16 12:02:45.898537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:45.900109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:46.021599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a89ccd-8fbc-4d38-b00d-20e176b7d4f3 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.026335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.027949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.028237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.028574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] PATH_INFO: `/v3/projects/97e42539fb1c4a089ebe8429c2f35f7c` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.029814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=97e42539fb1c4a089ebe8429c2f35f7c)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.031789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.161540 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '97e42539fb1c4a089ebe8429c2f35f7c', 'request_id': 'req-7c546d5c-5846-4ec1-bf03-00b8896df9e8'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:46.167760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '97e42539fb1c4a089ebe8429c2f35f7c', 'request_id': 'req-7c546d5c-5846-4ec1-bf03-00b8896df9e8'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:46.232872 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '97e42539fb1c4a089ebe8429c2f35f7c', 'request_id': 'req-7c546d5c-5846-4ec1-bf03-00b8896df9e8'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:46.248093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] The token cache is being invalidate because project 97e42539fb1c4a089ebe8429c2f35f7c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:02:46.248600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:46.249316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7c546d5c-5846-4ec1-bf03-00b8896df9e8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:46.251917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1259/2535] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:02:45 2022] DELETE /identity/v3/projects/97e42539fb1c4a089ebe8429c2f35f7c => generated 0 bytes in 353 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:46.256708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.257696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.258237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.278863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:46.280217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:46.352475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1277/2536] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:46.361880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.362178 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.362465 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.384583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:46.385829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:46.394348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1260/2537] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:46.398890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:46.409976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a14b04aa-6f1d-4cb9-ab87-a372e3033ccb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.414599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.415209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.415495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.415771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.416540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:02:46.416853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.418994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.435903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1278/2538] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:02:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 12:02:46.436677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:46.446320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5ad3cbd-deaa-4241-bd9e-c35dc1885df9 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.451237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.451944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.452235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.452511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.453212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.455177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.566624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1261/2539] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:46 2022] POST /identity/v3/projects => generated 381 bytes in 130 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:02:46.570896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:46.581467 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42729fda-06e7-4867-bc25-34e8f2939402 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.643135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.643694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.644062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.644449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.645106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.647385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.738790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1279/2540] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:46 2022] POST /identity/v3/users => generated 291 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:46.748312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:46.759922 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32654906-3a4d-4b2d-9335-25303ae01b3c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.777570 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.779479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.780042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.780352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.781135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.782800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.794967 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1262/2541] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:46.799088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:46.809733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ad00e48-7d09-48df-b342-95e3c19248ea admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.814909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e904049-0f1b-4eaf-a247-7a6e913e1b61 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.815587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e904049-0f1b-4eaf-a247-7a6e913e1b61 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.815902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e904049-0f1b-4eaf-a247-7a6e913e1b61 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.816559 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e904049-0f1b-4eaf-a247-7a6e913e1b61 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.816915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e904049-0f1b-4eaf-a247-7a6e913e1b61 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.818511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e904049-0f1b-4eaf-a247-7a6e913e1b61 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.842587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:46.843041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1280/2542] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:46 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:46.856166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d07379dc-c421-4b7f-8c25-b787369442b4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:46.861994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:46.869793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:46.869793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:46.869793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] PATH_INFO: `/v3/projects/50474a56df614a7e98466b3b04e52f4b/users/0b950e2869fa4842a8e97d2419897b59/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:46.888993 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] RBAC: Authorizing `identity:create_grant(project_id=50474a56df614a7e98466b3b04e52f4b, user_id=0b950e2869fa4842a8e97d2419897b59, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:46.892917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:46.896142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:47.029362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1263/2543] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:46 2022] PUT /identity/v3/projects/50474a56df614a7e98466b3b04e52f4b/users/0b950e2869fa4842a8e97d2419897b59/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:47.097357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.097638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.097915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.126427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] MFA Rules not processed for user `0b950e2869fa4842a8e97d2419897b59`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:47.127642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:47.138312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:47.197633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1281/2544] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:47 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:47.202959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:47.299491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16ed961e-971e-4a08-9d24-0c777b828733 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:47.304643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:47.305690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.306003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.306351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.306997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:47.309607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:47.435453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1264/2545] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:02:47 2022] POST /identity/v3/projects => generated 385 bytes in 233 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:47.439607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:47.451552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9006013b-5ef5-4ce9-a249-41c7abed1dcb None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:47.510022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:47.510696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.510982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.511830 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.512566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:47.514576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:47.601393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1282/2546] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:02:47 2022] POST /identity/v3/users => generated 299 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:02:47.605546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:47.616821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78188b14-763b-425e-aceb-fd308e218966 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:47.622183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:47.622857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.623144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.623425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.624307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:47.625841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:47.640617 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1265/2547] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:02:47.646933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:47.653679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb59e29-c488-443f-a1b1-62a9c8ccf62c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:47.659616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:47.660633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.660961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.661314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] PATH_INFO: `/v3/projects/94ecd4f3bc734f709920ddd9c92edbbe/users/e73b198d7efc40c091c229d60b13e1d1/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.677894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ecd4f3bc734f709920ddd9c92edbbe, user_id=e73b198d7efc40c091c229d60b13e1d1, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:47.680538 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:47.816999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1283/2548] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:47 2022] PUT /identity/v3/projects/94ecd4f3bc734f709920ddd9c92edbbe/users/e73b198d7efc40c091c229d60b13e1d1/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:47.821138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:47.832435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c828c922-fd45-41ea-8c27-0cadac9aae19 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:47.920793 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44f1998f-84a0-42b3-bc83-bcd15533731d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:47.921536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1998f-84a0-42b3-bc83-bcd15533731d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.921835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1998f-84a0-42b3-bc83-bcd15533731d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.922123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44f1998f-84a0-42b3-bc83-bcd15533731d admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.922909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1998f-84a0-42b3-bc83-bcd15533731d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:47.924718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44f1998f-84a0-42b3-bc83-bcd15533731d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:47.936986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1266/2549] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:02:47 2022] GET /identity/v3/roles => generated 1420 bytes in 116 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:02:47.941104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:47.953597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7777c309-f8c9-4179-b83d-f7f7e94be666 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:47.959796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:47.961061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:47.961523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:47.961984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] PATH_INFO: `/v3/projects/94ecd4f3bc734f709920ddd9c92edbbe/users/e73b198d7efc40c091c229d60b13e1d1/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:47.964803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] RBAC: Authorizing `identity:create_grant(project_id=94ecd4f3bc734f709920ddd9c92edbbe, user_id=e73b198d7efc40c091c229d60b13e1d1, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:47.967918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:47.998137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1284/2550] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:02:47 2022] PUT /identity/v3/projects/94ecd4f3bc734f709920ddd9c92edbbe/users/e73b198d7efc40c091c229d60b13e1d1/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:48.008914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:48.008914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:48.008914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:48.040610 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] MFA Rules not processed for user `e73b198d7efc40c091c229d60b13e1d1`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:48.042153 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:48.105577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:48.131921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1267/2551] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:48 2022] POST /identity/v3/auth/tokens => generated 3319 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:48.143634 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:48.144962 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:48.210077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe73c096-52c5-4705-bef6-56e0bc247506 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:48.213361 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:48.214026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:48.214849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:48.215134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:48.241994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:48.298577 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:48.301439 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:48.316903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1285/2552] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:48 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 174 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:53.558604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:53.569495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-54e41f47-aa10-450a-98f3-b50db701b327 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:53.575704 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c726770-99ee-48a4-b8ea-7cfb61497af1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:53.576418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726770-99ee-48a4-b8ea-7cfb61497af1 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:53.576770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726770-99ee-48a4-b8ea-7cfb61497af1 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:53.577122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c726770-99ee-48a4-b8ea-7cfb61497af1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:53.588242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c726770-99ee-48a4-b8ea-7cfb61497af1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:53.590481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c726770-99ee-48a4-b8ea-7cfb61497af1 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:53.607113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1268/2553] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:53 2022] GET /identity/v3/auth/tokens => generated 3319 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:02:58.310019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=19 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:02:59.392273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:59.462342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d93a948-af94-4ede-922d-4395486fdf01 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:59.468351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:59.469561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:59.469561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:59.469820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] PATH_INFO: `/v3/users/0b950e2869fa4842a8e97d2419897b59` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:59.470995 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0b950e2869fa4842a8e97d2419897b59)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:02:59.473235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:02:59.702313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0b950e2869fa4842a8e97d2419897b59'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:59.702990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0b950e2869fa4842a8e97d2419897b59'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:59.787247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0b950e2869fa4842a8e97d2419897b59'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:59.797353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0b950e2869fa4842a8e97d2419897b59'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:59.806956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0b950e2869fa4842a8e97d2419897b59'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:02:59.813156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1286/2554] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:02:59 2022] DELETE /identity/v3/users/0b950e2869fa4842a8e97d2419897b59 => generated 0 bytes in 421 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:02:59.817467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:59.817979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:59.818075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:02:59.836541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:02:59.837772 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:59.903588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1269/2555] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:02:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:02:59.913292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:02:59.914877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:02:59.985360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a9aa0fe-d13c-41cc-8790-6265e3de5943 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:02:59.989835 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:02:59.990324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:02:59.991973 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:02:59.992314 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:00.002607 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:00.004127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:00.014275 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1287/2556] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:02:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:00.394397 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:00.395850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:00.408041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc39d513-4ddc-4416-9c59-e7cc1b6fd902 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:00.412705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:00.413607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:00.413936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:00.414172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] PATH_INFO: `/v3/projects/50474a56df614a7e98466b3b04e52f4b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:00.415364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] RBAC: Authorizing `identity:delete_project(project_id=50474a56df614a7e98466b3b04e52f4b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:00.418215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:00.523303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '50474a56df614a7e98466b3b04e52f4b', 'request_id': 'req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.528630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '50474a56df614a7e98466b3b04e52f4b', 'request_id': 'req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.598647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '50474a56df614a7e98466b3b04e52f4b', 'request_id': 'req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.610046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] The token cache is being invalidate because project 50474a56df614a7e98466b3b04e52f4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:00.610334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.610973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-5a7181a4-2bee-4b4e-854e-f1bc464b9f71 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.612527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1270/2557] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:00 2022] DELETE /identity/v3/projects/50474a56df614a7e98466b3b04e52f4b => generated 0 bytes in 219 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:00.621135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:00.630637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a6d38cc-4c47-40e3-8dc3-ab84bb1d5e40 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:00.648630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:00.688253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:00.689134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:00.689418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:00.689694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] PATH_INFO: `/v3/users/e73b198d7efc40c091c229d60b13e1d1` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:00.691065 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] RBAC: Authorizing `identity:delete_user(user_id=e73b198d7efc40c091c229d60b13e1d1)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:00.693101 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:00.848317 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e73b198d7efc40c091c229d60b13e1d1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.848907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e73b198d7efc40c091c229d60b13e1d1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.853209 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:00.853697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:00.853802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:00.872104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] MFA Rules not processed for user `2e7985c45e3e478cac56a7d939e3fe2a`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:00.873302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:00.929436 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1271/2558] 10.222.0.40 () {58 vars in 967 bytes} [Thu Jun 16 12:03:00 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:03:00.936544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e73b198d7efc40c091c229d60b13e1d1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.942172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:00.945544 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:00.945673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e73b198d7efc40c091c229d60b13e1d1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.951040 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:00.959942 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e73b198d7efc40c091c229d60b13e1d1'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:00.968198 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1288/2559] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:00 2022] DELETE /identity/v3/users/e73b198d7efc40c091c229d60b13e1d1 => generated 0 bytes in 348 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:01.029165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46fd4ea7-7ea2-4f86-a8ef-d020f1e52bc8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:01.112361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '7740886489ae4fe78a1d81cd12852750', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:01.113156 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:01.113441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:01.113721 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:01.115553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:01.176991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:01.178595 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de49f2b9-8b43-4c91-a7c6-6cd1cb6ef2a0 service swift] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:01.190322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1272/2560] 10.222.0.40 () {60 vars in 1384 bytes} [Thu Jun 16 12:03:00 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 245 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jun 16 12:03:01.454407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:01.454407 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:01.523128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c180d293-bfaf-448b-9bc4-e1df408dad9a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:01.528479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:01.529404 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:01.529692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:01.529976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] PATH_INFO: `/v3/projects/94ecd4f3bc734f709920ddd9c92edbbe` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:01.531171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] RBAC: Authorizing `identity:delete_project(project_id=94ecd4f3bc734f709920ddd9c92edbbe)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:01.533136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:01.646585 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:01.649906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '94ecd4f3bc734f709920ddd9c92edbbe', 'request_id': 'req-04c54740-5e66-4e72-8df5-d5780707f3db'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:01.655978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '94ecd4f3bc734f709920ddd9c92edbbe', 'request_id': 'req-04c54740-5e66-4e72-8df5-d5780707f3db'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:01.661300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '94ecd4f3bc734f709920ddd9c92edbbe', 'request_id': 'req-04c54740-5e66-4e72-8df5-d5780707f3db'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:01.787650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] The token cache is being invalidate because project 94ecd4f3bc734f709920ddd9c92edbbe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:01.788212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:01.788956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-04c54740-5e66-4e72-8df5-d5780707f3db admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:01.790646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1289/2561] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:01 2022] DELETE /identity/v3/projects/94ecd4f3bc734f709920ddd9c92edbbe => generated 0 bytes in 339 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:01.796715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23607871-15a7-4209-907e-412ccb161b92 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:01.797032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23607871-15a7-4209-907e-412ccb161b92 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:01.797312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23607871-15a7-4209-907e-412ccb161b92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:01.815198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-23607871-15a7-4209-907e-412ccb161b92 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:01.816444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-23607871-15a7-4209-907e-412ccb161b92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:01.863052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=100 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:01.879316 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1273/2562] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:01.888625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:01.888961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:01.889312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:01.907574 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:01.908816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:01.917496 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1290/2563] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:01.921667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23607871-15a7-4209-907e-412ccb161b92 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:01.931149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23607871-15a7-4209-907e-412ccb161b92 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:01.936558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:01.937214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:01.937503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:01.937782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:01.938511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:03:01.938824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:01.940501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:01.955780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1274/2564] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:03:01 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:03:01.958206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:01.966839 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ac39ac-29ce-4d9f-9112-fca773dd6452 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:01.971980 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:01.972897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:01.973177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:01.973460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:01.974148 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:01.976068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:02.105836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1291/2565] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:01 2022] POST /identity/v3/projects => generated 375 bytes in 148 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:02.109443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:02.120948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8afe369d-8ed5-4557-96f2-0ee9ca571b13 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:02.210830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:02.211789 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:02.212014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:02.212282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:02.212988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:02.215722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:02.312937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1275/2566] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:02 2022] POST /identity/v3/users => generated 288 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:02.316010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:02.327099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b5c5d7-1afa-4ec7-aa86-b8653b59bac7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:02.339990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:02.339990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:02.339990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:02.339990 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:02.343372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:02.343372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:02.349127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1292/2567] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:02.353137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:02.364693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b296f6ae-80e1-4d9b-8fc0-954842424bd6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:02.369645 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0a5bfd-8a19-4f6a-8712-d39301d05915 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:02.370332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a5bfd-8a19-4f6a-8712-d39301d05915 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:02.370738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a5bfd-8a19-4f6a-8712-d39301d05915 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:02.371131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0a5bfd-8a19-4f6a-8712-d39301d05915 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:02.371736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a5bfd-8a19-4f6a-8712-d39301d05915 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:02.373357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd0a5bfd-8a19-4f6a-8712-d39301d05915 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:02.384163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1276/2568] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:02 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:02.388592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:02.398582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c5a92fd-4a03-43ab-95b3-3ea0bea3db0b admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:02.403935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478755-3187-4341-9111-9aac0926715e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:02.404721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478755-3187-4341-9111-9aac0926715e admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:02.405014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478755-3187-4341-9111-9aac0926715e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:02.405299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90478755-3187-4341-9111-9aac0926715e admin admin] PATH_INFO: `/v3/projects/8bfd116c0cd64285a2fc25b191b941dc/users/066a21b7951e4e8ea1004288d85c12e5/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:02.428351 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90478755-3187-4341-9111-9aac0926715e admin admin] RBAC: Authorizing `identity:create_grant(project_id=8bfd116c0cd64285a2fc25b191b941dc, user_id=066a21b7951e4e8ea1004288d85c12e5, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:02.433554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90478755-3187-4341-9111-9aac0926715e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:02.536389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1293/2569] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:02 2022] PUT /identity/v3/projects/8bfd116c0cd64285a2fc25b191b941dc/users/066a21b7951e4e8ea1004288d85c12e5/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:02.548307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:02.548600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:02.548929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:02.576577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] MFA Rules not processed for user `066a21b7951e4e8ea1004288d85c12e5`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:02.577809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:02.650977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1277/2570] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:02 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:02.662688 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478755-3187-4341-9111-9aac0926715e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:02.664434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-90478755-3187-4341-9111-9aac0926715e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:02.725267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90478755-3187-4341-9111-9aac0926715e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:02.729042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97af721a-de6f-4846-af87-8e86ebbfc1fb service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:02.729728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af721a-de6f-4846-af87-8e86ebbfc1fb service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:02.730014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af721a-de6f-4846-af87-8e86ebbfc1fb service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:02.730296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97af721a-de6f-4846-af87-8e86ebbfc1fb service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:02.738671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af721a-de6f-4846-af87-8e86ebbfc1fb service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:02.740173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97af721a-de6f-4846-af87-8e86ebbfc1fb service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:02.749183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1294/2571] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:02 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:06.740170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:06.786542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:06.804954 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b15cf30-9825-49a7-ac92-08d00aaa061d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:06.809727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:06.810607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:06.810890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:06.811166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] PATH_INFO: `/v3/users/066a21b7951e4e8ea1004288d85c12e5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:06.812453 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=066a21b7951e4e8ea1004288d85c12e5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:06.814418 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:07.064160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '066a21b7951e4e8ea1004288d85c12e5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:07.064820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '066a21b7951e4e8ea1004288d85c12e5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:07.136673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '066a21b7951e4e8ea1004288d85c12e5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:07.144047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '066a21b7951e4e8ea1004288d85c12e5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:07.156219 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '066a21b7951e4e8ea1004288d85c12e5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:07.162058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1278/2572] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:06 2022] DELETE /identity/v3/users/066a21b7951e4e8ea1004288d85c12e5 => generated 0 bytes in 422 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:07.166739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:07.167243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:07.167447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:07.184292 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:07.185500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:07.244731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1295/2573] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:07.254499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:07.255949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:07.311338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f88f3b47-0791-40f8-bbc3-0ce81e947ce4 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:07.314924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:07.315502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:07.315784 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:07.316098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:07.325384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:07.326801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:07.336013 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1279/2574] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:07.868629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:07.871643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:07.885166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0d7dea-973a-47c6-a261-baef6e3720f6 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:07.890556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:07.891454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:07.891738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:07.892061 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] PATH_INFO: `/v3/projects/8bfd116c0cd64285a2fc25b191b941dc` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:07.893294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] RBAC: Authorizing `identity:delete_project(project_id=8bfd116c0cd64285a2fc25b191b941dc)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:07.895208 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:07.902698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:DELETE=4,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:08.016166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8bfd116c0cd64285a2fc25b191b941dc', 'request_id': 'req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:08.022309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8bfd116c0cd64285a2fc25b191b941dc', 'request_id': 'req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:08.028648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8bfd116c0cd64285a2fc25b191b941dc', 'request_id': 'req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:08.112898 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] The token cache is being invalidate because project 8bfd116c0cd64285a2fc25b191b941dc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:08.113321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:08.113984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:08.115534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1296/2575] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:07 2022] DELETE /identity/v3/projects/8bfd116c0cd64285a2fc25b191b941dc => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:12.293326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:12.294723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:12.305731 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8815657e-c8b7-41e6-a915-99f2085c057c service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:12.353802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:12.355001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:12.355289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:12.355571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:12.357353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:12.376735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:12.419297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:12.420814 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:12.431139 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1280/2576] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 138 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:15.894419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:15.895848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:15.970245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6aafafc9-f2e9-4adf-8cb6-9da252bfddfc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:15.975526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:15.979976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:15.979976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:15.979976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:15.979976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:15.990059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:15.991669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:16.002619 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1297/2577] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:15 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:16.011941 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:16.014065 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:16.026831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d600294-1da4-4f35-a85c-f15fcfcbc111 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:16.031485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:16.032396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:16.032683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:16.032960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] PATH_INFO: `/v3/users/13d8c04611e24dffad16f8bb97e111f2` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:16.034145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=13d8c04611e24dffad16f8bb97e111f2)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:16.036112 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:16.229110 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '13d8c04611e24dffad16f8bb97e111f2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.230254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '13d8c04611e24dffad16f8bb97e111f2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.290057 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '13d8c04611e24dffad16f8bb97e111f2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.297691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '13d8c04611e24dffad16f8bb97e111f2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.310214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7824be46-a19c-4c47-ba1d-63112ce04fc4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '13d8c04611e24dffad16f8bb97e111f2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.316207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1281/2578] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:16 2022] DELETE /identity/v3/users/13d8c04611e24dffad16f8bb97e111f2 => generated 0 bytes in 306 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:16.554135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:16.566570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e45b858-148b-4357-950c-f6aa265742c8 service placement] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:16.624976 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:16.625836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:16.626119 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:16.626392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] PATH_INFO: `/v3/projects/918001434f5c4d76a3ab9c46616eec67` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:16.627563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] RBAC: Authorizing `identity:delete_project(project_id=918001434f5c4d76a3ab9c46616eec67)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:16.629491 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:16.743150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:16.758765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '918001434f5c4d76a3ab9c46616eec67', 'request_id': 'req-62867fde-ed0f-40b5-9e5c-2f84cc074792'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.764740 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '918001434f5c4d76a3ab9c46616eec67', 'request_id': 'req-62867fde-ed0f-40b5-9e5c-2f84cc074792'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.770167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '918001434f5c4d76a3ab9c46616eec67', 'request_id': 'req-62867fde-ed0f-40b5-9e5c-2f84cc074792'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.859485 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] The token cache is being invalidate because project 918001434f5c4d76a3ab9c46616eec67 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:16.859778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.860449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-62867fde-ed0f-40b5-9e5c-2f84cc074792 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:16.862329 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1298/2579] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:16 2022] DELETE /identity/v3/projects/918001434f5c4d76a3ab9c46616eec67 => generated 0 bytes in 309 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:16.867357 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:16.867648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:16.867948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:16.885130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:16.886838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:16.944737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1282/2580] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:16.953136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:16.953625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:16.953723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:16.972172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:16.973647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:16.981562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1299/2581] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:16.985652 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:16.995272 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-087e8750-dcfb-462e-a290-deef5313fd1e None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.000268 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.000921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.001207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.001479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.002208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:03:17.002516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.004132 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.016127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1283/2582] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:03:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:03:17.020338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.030139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86879f33-fc6b-414c-9e18-646c006e3348 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.034854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.035508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.035798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.036110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.036801 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.038699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.124569 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1300/2583] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:17 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:17.128596 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.139028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a4a4404-8f2d-41e6-9a8e-af8a0d37a221 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.150174 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:17.193378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.194046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.194338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.194616 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.195349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.197330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.279080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1284/2584] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:17 2022] POST /identity/v3/users => generated 294 bytes in 151 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:17.283207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.293595 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdcc1442-ae6a-44ee-b253-bcfd92d010e6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.298241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.298900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.299187 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.299483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.300281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.301860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.312514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1301/2585] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:17.326726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.336749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-048338ec-0519-4fbe-83c9-4806fc364c51 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.341450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180facc2-8e0d-43f6-8739-70885185b2e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.346785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180facc2-8e0d-43f6-8739-70885185b2e3 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.346785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180facc2-8e0d-43f6-8739-70885185b2e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.346785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180facc2-8e0d-43f6-8739-70885185b2e3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.346785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180facc2-8e0d-43f6-8739-70885185b2e3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.346785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180facc2-8e0d-43f6-8739-70885185b2e3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.355378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1285/2586] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:17.359554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.369841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706b7b6f-034f-4077-bee0-1c7a5c035a1a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.375073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.375984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.376295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.376572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] PATH_INFO: `/v3/projects/68ef09d660b64ee885821ba82c642824/users/937a4c6b13bb4908a5bc4a9bbdd46ee9/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.392936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] RBAC: Authorizing `identity:create_grant(project_id=68ef09d660b64ee885821ba82c642824, user_id=937a4c6b13bb4908a5bc4a9bbdd46ee9, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.395669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.477527 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1302/2587] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:17 2022] PUT /identity/v3/projects/68ef09d660b64ee885821ba82c642824/users/937a4c6b13bb4908a5bc4a9bbdd46ee9/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:17.488637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.488929 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.489208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.516441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] MFA Rules not processed for user `937a4c6b13bb4908a5bc4a9bbdd46ee9`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:17.517600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:17.580090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1286/2588] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:17.584523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.645722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac006a1-3caa-462c-9225-d5e871bc816a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.650561 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.651220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.651509 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.651789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.652547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.654441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.765166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1303/2589] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:17 2022] POST /identity/v3/projects => generated 387 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:03:17.769276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.779803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5339cec-132a-416b-865f-c17f1c55087b None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.836078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.836760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.837052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.837329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.838032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.840086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.896682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1287/2590] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:17 2022] POST /identity/v3/users => generated 294 bytes in 128 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:17.900734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.911429 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-006d71f4-8dc6-4c54-b319-da381cce8c00 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.916162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.916825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.917629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.917910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.918644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.920295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.930745 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1304/2591] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:17.935368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.945375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb343a76-774d-479a-8f34-972cd2f8f3ba admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.949951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89479007-3913-4861-9cf7-d14fe31ad378 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.953620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89479007-3913-4861-9cf7-d14fe31ad378 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.953620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89479007-3913-4861-9cf7-d14fe31ad378 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.953620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89479007-3913-4861-9cf7-d14fe31ad378 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.953620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89479007-3913-4861-9cf7-d14fe31ad378 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:17.953620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89479007-3913-4861-9cf7-d14fe31ad378 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:17.963175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1288/2592] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:17 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:17.967624 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:17.977853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75129f53-b9c5-47a8-a41c-439cbbff6084 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:17.983026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:17.983919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:17.984207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:17.984484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] PATH_INFO: `/v3/projects/795edfcc3af143568e033325de83cef5/users/2547784d91dc41e8ab49cc1b9787ac5c/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:17.998605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] RBAC: Authorizing `identity:create_grant(project_id=795edfcc3af143568e033325de83cef5, user_id=2547784d91dc41e8ab49cc1b9787ac5c, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.001265 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.080145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:DELETE=6,keystone:SELECT=89,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:18.084770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1305/2593] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:17 2022] PUT /identity/v3/projects/795edfcc3af143568e033325de83cef5/users/2547784d91dc41e8ab49cc1b9787ac5c/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:18.095334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.095826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.095982 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.102012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:18.122927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] MFA Rules not processed for user `2547784d91dc41e8ab49cc1b9787ac5c`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:18.124167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:18.187717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1289/2594] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:18 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:18.192260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.256177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91daee94-97c0-4f02-89d2-7b45121e4757 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.260819 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.261552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.261837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.262115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.262822 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.264760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.325891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1306/2595] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:18 2022] POST /identity/v3/projects => generated 387 bytes in 134 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:18.330109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.341586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72659f3a-3fac-49cf-bc50-bda96b1007c0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.396921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.397611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.397937 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.398222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.399000 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.401003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.461273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1290/2596] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:18 2022] POST /identity/v3/users => generated 300 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:18.466010 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.476611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-873800d2-6869-41d8-9d65-327e72da22bd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.481237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.482037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.482331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.482639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.483376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.485040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.496287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1307/2597] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:18.501904 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.510529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bea39513-7fb1-445f-9d60-bd2977672436 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.516015 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.516926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.517216 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.517493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] PATH_INFO: `/v3/projects/bbd97e6ec19d444fbeb8d655d8ec3447/users/ed82dda384334cb4b99f2139c5ebd99e/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.532744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd97e6ec19d444fbeb8d655d8ec3447, user_id=ed82dda384334cb4b99f2139c5ebd99e, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.535389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.586288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1291/2598] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:18 2022] PUT /identity/v3/projects/bbd97e6ec19d444fbeb8d655d8ec3447/users/ed82dda384334cb4b99f2139c5ebd99e/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 86 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:18.590804 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.601449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27506121-28ea-46f4-a29f-0f1741d96dc1 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.657428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9698848-9e60-4542-a778-cfb7f0360a3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.658094 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698848-9e60-4542-a778-cfb7f0360a3f admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.658382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698848-9e60-4542-a778-cfb7f0360a3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.658660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9698848-9e60-4542-a778-cfb7f0360a3f admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.659384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698848-9e60-4542-a778-cfb7f0360a3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.661005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9698848-9e60-4542-a778-cfb7f0360a3f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.670758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1308/2599] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:18 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:18.675023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.686123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4f4c70-01c0-4f83-bed7-a38c7d21804e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.690803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.691694 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.692034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.692319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] PATH_INFO: `/v3/projects/bbd97e6ec19d444fbeb8d655d8ec3447/users/ed82dda384334cb4b99f2139c5ebd99e/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.694334 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] RBAC: Authorizing `identity:create_grant(project_id=bbd97e6ec19d444fbeb8d655d8ec3447, user_id=ed82dda384334cb4b99f2139c5ebd99e, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.697017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.708012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1292/2600] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:18 2022] PUT /identity/v3/projects/bbd97e6ec19d444fbeb8d655d8ec3447/users/ed82dda384334cb4b99f2139c5ebd99e/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:18.718130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.718427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.718712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.744442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] MFA Rules not processed for user `ed82dda384334cb4b99f2139c5ebd99e`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:18.746127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:18.803362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:18.812606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1309/2601] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:18.823087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:18.825380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:18.870999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:18.881222 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b988a4b-6f2b-48f9-9369-d8771cc0327a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:18.884873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:18.885549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:18.886424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:18.886700 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:18.895682 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:18.897180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:18.906802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1293/2602] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:19.926654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:19.928127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:19.940028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b737f2-ac40-4110-b5e2-329f91af2848 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:19.943562 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:19.944239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:19.944526 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:19.944802 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:20.000487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:20.001916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:20.011135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1310/2603] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:19 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:24.563335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:24.564023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e267e63-e842-474c-b4f8-4fbbce3d9894 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:24.567673 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9372a6ba-68b8-40db-8998-3904011552ee service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:24.568376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372a6ba-68b8-40db-8998-3904011552ee service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:24.568666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372a6ba-68b8-40db-8998-3904011552ee service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:24.568949 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9372a6ba-68b8-40db-8998-3904011552ee service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:24.625195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372a6ba-68b8-40db-8998-3904011552ee service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:24.626619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9372a6ba-68b8-40db-8998-3904011552ee service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:24.636122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1294/2604] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:24 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:28.228500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:28.289426 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87dd20bb-cd9a-48ec-b0a5-9f5467618d9b service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:28.294411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:28.295276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:28.295563 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:28.295836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] PATH_INFO: `/v3/users/937a4c6b13bb4908a5bc4a9bbdd46ee9` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:28.297027 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=937a4c6b13bb4908a5bc4a9bbdd46ee9)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:28.298948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:28.463045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '937a4c6b13bb4908a5bc4a9bbdd46ee9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:28.464225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '937a4c6b13bb4908a5bc4a9bbdd46ee9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:28.545928 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '937a4c6b13bb4908a5bc4a9bbdd46ee9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:28.552892 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '937a4c6b13bb4908a5bc4a9bbdd46ee9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:28.564601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '937a4c6b13bb4908a5bc4a9bbdd46ee9'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:28.570402 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1311/2605] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:28 2022] DELETE /identity/v3/users/937a4c6b13bb4908a5bc4a9bbdd46ee9 => generated 0 bytes in 342 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:28.575217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:28.575697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:28.575815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:28.593079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:28.594278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:28.653127 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1295/2606] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:28.662435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:28.663841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:28.679596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:28.722582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2b87a4f-e239-4844-8f3a-a490dc9e75d0 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:28.726861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:28.727506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:28.727792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:28.728100 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:28.737536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:28.738958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:28.748230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1312/2607] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:29.142898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:29.144203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:29.157968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd944ee2-0cff-4523-bf31-2fb753fb4140 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:29.162696 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:29.165722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:29.165722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:29.165722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] PATH_INFO: `/v3/projects/68ef09d660b64ee885821ba82c642824` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:29.165722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] RBAC: Authorizing `identity:delete_project(project_id=68ef09d660b64ee885821ba82c642824)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:29.167428 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:29.266559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '68ef09d660b64ee885821ba82c642824', 'request_id': 'req-6758b737-864a-4a6b-bb39-b6956ebddced'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.271792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '68ef09d660b64ee885821ba82c642824', 'request_id': 'req-6758b737-864a-4a6b-bb39-b6956ebddced'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.324699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '68ef09d660b64ee885821ba82c642824', 'request_id': 'req-6758b737-864a-4a6b-bb39-b6956ebddced'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.335751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] The token cache is being invalidate because project 68ef09d660b64ee885821ba82c642824 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:29.336534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.336703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.338242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1296/2608] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:29 2022] DELETE /identity/v3/projects/68ef09d660b64ee885821ba82c642824 => generated 0 bytes in 196 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:29.342199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:29.352207 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a29a37-9eda-433c-9227-52c7f638acf7 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:29.407573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:29.408473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:29.408763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:29.409043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] PATH_INFO: `/v3/users/2547784d91dc41e8ab49cc1b9787ac5c` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:29.410214 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] RBAC: Authorizing `identity:delete_user(user_id=2547784d91dc41e8ab49cc1b9787ac5c)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:29.412151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:29.561147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '2547784d91dc41e8ab49cc1b9787ac5c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.561753 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '2547784d91dc41e8ab49cc1b9787ac5c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.621731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '2547784d91dc41e8ab49cc1b9787ac5c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.630195 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '2547784d91dc41e8ab49cc1b9787ac5c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.642134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '2547784d91dc41e8ab49cc1b9787ac5c'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:29.648410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1313/2609] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:29 2022] DELETE /identity/v3/users/2547784d91dc41e8ab49cc1b9787ac5c => generated 0 bytes in 307 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:30.051191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:30.052630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:30.067288 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6758b737-864a-4a6b-bb39-b6956ebddced admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:30.072641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:30.123948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:30.124813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:30.125099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:30.125375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] PATH_INFO: `/v3/projects/795edfcc3af143568e033325de83cef5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:30.126553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] RBAC: Authorizing `identity:delete_project(project_id=795edfcc3af143568e033325de83cef5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:30.128523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:30.226948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '795edfcc3af143568e033325de83cef5', 'request_id': 'req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.232861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '795edfcc3af143568e033325de83cef5', 'request_id': 'req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.278311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '795edfcc3af143568e033325de83cef5', 'request_id': 'req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.289469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] The token cache is being invalidate because project 795edfcc3af143568e033325de83cef5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:30.289761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.290431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.292593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1297/2610] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:30 2022] DELETE /identity/v3/projects/795edfcc3af143568e033325de83cef5 => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:30.296805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:30.298206 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:30.310935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd19d41-1290-4774-b8f7-1e129e7cb468 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:30.343389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:30.366916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:30.367779 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:30.368091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:30.368372 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] PATH_INFO: `/v3/users/ed82dda384334cb4b99f2139c5ebd99e` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:30.369530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=ed82dda384334cb4b99f2139c5ebd99e)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:30.371422 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:30.508670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ed82dda384334cb4b99f2139c5ebd99e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.509328 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ed82dda384334cb4b99f2139c5ebd99e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.564056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ed82dda384334cb4b99f2139c5ebd99e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.571294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ed82dda384334cb4b99f2139c5ebd99e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.583673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-78af5913-3a42-47ef-8d94-63755d8dadfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ed82dda384334cb4b99f2139c5ebd99e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:30.590464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1314/2611] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:30 2022] DELETE /identity/v3/users/ed82dda384334cb4b99f2139c5ebd99e => generated 0 bytes in 294 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:31.012161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.013591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:31.026777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-997bc31e-c0a6-4cb7-85a2-f26fbe03bdcc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.084344 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.085210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.085495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.085769 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] PATH_INFO: `/v3/projects/bbd97e6ec19d444fbeb8d655d8ec3447` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.086969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] RBAC: Authorizing `identity:delete_project(project_id=bbd97e6ec19d444fbeb8d655d8ec3447)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.088965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.175741 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:31.181477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'bbd97e6ec19d444fbeb8d655d8ec3447', 'request_id': 'req-42a2e091-2eef-46a2-b29c-9d0b467beb4a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:31.187396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'bbd97e6ec19d444fbeb8d655d8ec3447', 'request_id': 'req-42a2e091-2eef-46a2-b29c-9d0b467beb4a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:31.266605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'bbd97e6ec19d444fbeb8d655d8ec3447', 'request_id': 'req-42a2e091-2eef-46a2-b29c-9d0b467beb4a'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:31.277681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] The token cache is being invalidate because project bbd97e6ec19d444fbeb8d655d8ec3447 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:31.278165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:31.279275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-42a2e091-2eef-46a2-b29c-9d0b467beb4a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:31.280957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1298/2612] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:31 2022] DELETE /identity/v3/projects/bbd97e6ec19d444fbeb8d655d8ec3447 => generated 0 bytes in 270 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:31.287903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.288201 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.288513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.307079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:31.308927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:31.365629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1315/2613] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:31.374530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.374825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.375107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.392968 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:31.394166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:31.401936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1299/2614] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:31.405881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.415387 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39bca511-388c-4e1c-8462-b3fdc6c6ac06 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.420583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.421244 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.421536 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.421816 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.422579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:03:31.422891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.424534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.436814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1316/2615] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:03:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:03:31.440770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.450078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5524cec5-dc01-409f-aba7-94f8a60a2e0c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.455035 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.455699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.456025 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.456308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.457173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.459054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.548672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1300/2616] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:31 2022] POST /identity/v3/projects => generated 373 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:03:31.552927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.563072 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec5ee80-e9f9-43db-a6eb-1e915c23694e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.602796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:31.618782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.619451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.619749 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.620053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.620739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.622652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.687759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1317/2617] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:31 2022] POST /identity/v3/users => generated 287 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:03:31.688921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.699686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8770bf61-b1bb-450e-a5cd-ef24e3fb3aad admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.704671 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.705337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.705630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.705913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.706705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.708424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.719292 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1301/2618] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:31.723947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.734411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9630d4d-9c32-4d9d-bc0a-b937fda8e2f5 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.739523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b8cd8a1-0170-43d9-86fe-5fbcf20e9ed5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.740203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cd8a1-0170-43d9-86fe-5fbcf20e9ed5 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.740488 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cd8a1-0170-43d9-86fe-5fbcf20e9ed5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.740763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8cd8a1-0170-43d9-86fe-5fbcf20e9ed5 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.741480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cd8a1-0170-43d9-86fe-5fbcf20e9ed5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.743067 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b8cd8a1-0170-43d9-86fe-5fbcf20e9ed5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.752790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1318/2619] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:31 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:31.756686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.766637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15ebf6d4-2492-4c5e-aa2c-2671848e593d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:31.771315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:31.772228 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.772519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.772796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] PATH_INFO: `/v3/projects/3acb29520eb94e5a978e31f0b2ac0b1e/users/1cd38482beb94f7a94d9e33376424d67/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.787355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3acb29520eb94e5a978e31f0b2ac0b1e, user_id=1cd38482beb94f7a94d9e33376424d67, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:31.790045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:31.860431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1302/2620] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:31 2022] PUT /identity/v3/projects/3acb29520eb94e5a978e31f0b2ac0b1e/users/1cd38482beb94f7a94d9e33376424d67/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:31.865551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e0c8-43dd-4efb-8155-a7437b7be965 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:31.865844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e0c8-43dd-4efb-8155-a7437b7be965 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:31.866157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d911e0c8-43dd-4efb-8155-a7437b7be965 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:31.884147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-d911e0c8-43dd-4efb-8155-a7437b7be965 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:31.885848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d911e0c8-43dd-4efb-8155-a7437b7be965 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:31.943455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1319/2621] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:03:31.953441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:31.955086 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:32.011218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52ac067b-cb68-4807-9fb4-532ee7495dd3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:32.014879 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:32.015518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:32.015804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:32.016115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:32.029017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:32.030439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:32.039914 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1303/2622] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:31 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:33.089332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.089875 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.090081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.119843 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] MFA Rules not processed for user `1cd38482beb94f7a94d9e33376424d67`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:33.121083 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:33.191540 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1320/2623] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:33 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:33.196278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:33.206117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5d12728-bdd7-42ff-ac96-320077ddc9fe service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:33.210751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:33.211400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.211681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.211997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.212727 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:33.215158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:33.320303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1304/2624] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:33 2022] POST /identity/v3/projects => generated 373 bytes in 124 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:33.324854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:33.335829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df76ffa2-58c7-4b13-8df3-7b2dc51bb539 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:33.395556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:33.405581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:33.406259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.406579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.406861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.407551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:33.409770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:33.507268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1321/2625] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:33 2022] POST /identity/v3/users => generated 293 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:33.512281 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:33.522997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7550d15f-f36d-456f-be52-b7f8826a23d6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:33.528380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:33.530271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.530271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.530271 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.530915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:33.532087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:33.543158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1305/2626] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:33.547444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:33.557842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76a93fc6-7cdb-47cd-85b7-05b2e67e8c80 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:33.562918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:33.563977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.564339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.564776 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] PATH_INFO: `/v3/projects/04e27a4290da4fe7947b95cda6467f39/users/e5817d7f7f8640a88edb1136e93e3f43/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.583161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e27a4290da4fe7947b95cda6467f39, user_id=e5817d7f7f8640a88edb1136e93e3f43, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:33.585847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:33.638863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1322/2627] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:33 2022] PUT /identity/v3/projects/04e27a4290da4fe7947b95cda6467f39/users/e5817d7f7f8640a88edb1136e93e3f43/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 92 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:33.642980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:33.653541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bf5719b-6349-4200-aa28-ce911823a6a3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:33.687154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:33.711184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0308d992-756a-409e-bf76-9345f80882cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:33.712010 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308d992-756a-409e-bf76-9345f80882cc admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.712303 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308d992-756a-409e-bf76-9345f80882cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.712581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0308d992-756a-409e-bf76-9345f80882cc admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.713301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308d992-756a-409e-bf76-9345f80882cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:33.714894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0308d992-756a-409e-bf76-9345f80882cc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:33.724966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1306/2628] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:33 2022] GET /identity/v3/roles => generated 1420 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:33.729077 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:33.739135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6605ef4d-856b-45a6-9a00-418d29ff0a47 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:33.743752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:33.745231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:33.745507 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:33.745788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] PATH_INFO: `/v3/projects/04e27a4290da4fe7947b95cda6467f39/users/e5817d7f7f8640a88edb1136e93e3f43/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:33.747762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=04e27a4290da4fe7947b95cda6467f39, user_id=e5817d7f7f8640a88edb1136e93e3f43, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:33.750447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:33.761593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1323/2629] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:33 2022] PUT /identity/v3/projects/04e27a4290da4fe7947b95cda6467f39/users/e5817d7f7f8640a88edb1136e93e3f43/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:34.871229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:34.871777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:34.871777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:34.903608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] MFA Rules not processed for user `e5817d7f7f8640a88edb1136e93e3f43`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:34.909115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:34.989625 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1307/2630] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:34 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:35.001032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:35.002882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:35.061890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b493ada0-cd23-46dc-8d4b-3e07829399a8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:35.065480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:35.066125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:35.066433 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:35.066710 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:35.125055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:35.126493 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:35.135943 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1324/2631] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:34 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 135 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:36.956763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:36.958194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:36.970507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a487b45d-be65-472d-a210-0555a4154e07 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:36.976657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:36.976657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:36.976657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:36.976657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:36.985981 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:36.987486 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:36.998011 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1308/2632] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:36 2022] GET /identity/v3/auth/tokens => generated 3307 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:40.389480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:40.446023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a58652d8-4b92-49e6-a25b-6d5c26f1f19a service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:40.450256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:40.450903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:40.451194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:40.451474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:40.453279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:40.463151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:40.517021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:40.518984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:40.529423 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1325/2633] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 141 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:44.125496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:44.127150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:44.141960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fb8b7d2-0edc-4e8a-8c1d-97a7c341eeba service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:44.146584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:44.147450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:44.147734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:44.148044 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] PATH_INFO: `/v3/users/97ce6d944eb74b9794d6d0d5f18d4bad` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:44.149224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=97ce6d944eb74b9794d6d0d5f18d4bad)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:44.151133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:44.274098 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97ce6d944eb74b9794d6d0d5f18d4bad'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.274841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97ce6d944eb74b9794d6d0d5f18d4bad'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.318148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97ce6d944eb74b9794d6d0d5f18d4bad'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.324944 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97ce6d944eb74b9794d6d0d5f18d4bad'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.336412 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-1de6fb7d-c605-4cdd-b483-c4ed9595fc78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97ce6d944eb74b9794d6d0d5f18d4bad'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.342852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1309/2634] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:44 2022] DELETE /identity/v3/users/97ce6d944eb74b9794d6d0d5f18d4bad => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:44.588479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:44.600224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8fa3441-6d88-4f29-8f37-0a63bb5ec334 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:44.660141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:44.661014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:44.661300 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:44.661587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] PATH_INFO: `/v3/projects/659db25373614c8da6fc81e59ef9c6f6` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:44.662765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=659db25373614c8da6fc81e59ef9c6f6)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:44.664723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:44.782398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '659db25373614c8da6fc81e59ef9c6f6', 'request_id': 'req-71c9344f-a93c-4f5c-beee-f9f504eb5db4'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.788495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '659db25373614c8da6fc81e59ef9c6f6', 'request_id': 'req-71c9344f-a93c-4f5c-beee-f9f504eb5db4'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.794149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '659db25373614c8da6fc81e59ef9c6f6', 'request_id': 'req-71c9344f-a93c-4f5c-beee-f9f504eb5db4'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.890056 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] The token cache is being invalidate because project 659db25373614c8da6fc81e59ef9c6f6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:44.890270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.890882 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-71c9344f-a93c-4f5c-beee-f9f504eb5db4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:44.892449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1326/2635] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:44 2022] DELETE /identity/v3/projects/659db25373614c8da6fc81e59ef9c6f6 => generated 0 bytes in 305 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:44.897332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:44.897619 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:44.897891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:44.914935 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:44.916161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:44.925313 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:44.976393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1310/2636] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:03:44.985032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:44.985356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:44.985780 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.004419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:45.005608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:45.014044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1327/2637] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:45.018115 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.028637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e7dcac8-066a-4c8c-8bc4-a32e47e6fa51 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.033396 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.034067 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.034381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.034660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.035393 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:03:45.035703 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.037487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.050307 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1311/2638] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:03:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:03:45.054733 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.064919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-656938bc-24e3-4e4d-b748-25e9383b9af5 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.070097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.070758 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.071049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.071325 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.072053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.076789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.186706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1328/2639] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:45 2022] POST /identity/v3/projects => generated 383 bytes in 129 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:45.187971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.197882 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea6bc6e-7ac3-449a-84ec-21a12f767dd9 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.255167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.255837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.256151 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.256425 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.257104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.259006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.427389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1312/2640] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:45 2022] POST /identity/v3/users => generated 292 bytes in 240 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:45.431356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.442147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c37e8829-9982-424c-93fb-6a7a58eeaffb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.446719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.447381 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.447671 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.447991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.448791 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.450403 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.452483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:45.461305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1329/2641] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:45.465084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.475330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55382fcf-fb14-4627-84a8-1dcaa88bf2dd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.480093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd17968b-3e20-450b-a93d-93763966532f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.481251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17968b-3e20-450b-a93d-93763966532f admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.481543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17968b-3e20-450b-a93d-93763966532f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.481810 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd17968b-3e20-450b-a93d-93763966532f admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.482577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17968b-3e20-450b-a93d-93763966532f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.484191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd17968b-3e20-450b-a93d-93763966532f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.494325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1313/2642] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:45 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:45.498999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.509005 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b27f3f5-1c39-4fec-93a4-abfc09fffe39 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.514353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.515230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.515519 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.515800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] PATH_INFO: `/v3/projects/4e4082fe2ecb4756bdc57454c80d3b40/users/61dcabaa88464ba2aee72c163414d327/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.530028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e4082fe2ecb4756bdc57454c80d3b40, user_id=61dcabaa88464ba2aee72c163414d327, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.532728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.580991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1330/2643] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:45 2022] PUT /identity/v3/projects/4e4082fe2ecb4756bdc57454c80d3b40/users/61dcabaa88464ba2aee72c163414d327/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 82 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:45.585972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce0271-7c24-4a89-a704-2b3c46a6b279 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.586265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce0271-7c24-4a89-a704-2b3c46a6b279 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.586545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbce0271-7c24-4a89-a704-2b3c46a6b279 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.604186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-bbce0271-7c24-4a89-a704-2b3c46a6b279 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:45.605939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bbce0271-7c24-4a89-a704-2b3c46a6b279 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:45.663805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1314/2644] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:45.674369 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:45.676156 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:45.735019 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7aa13a8d-9024-4fc6-b913-cdda62f2d31e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:45.738762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:45.739411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:45.739715 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:45.740073 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:45.749291 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:45.750786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:45.760676 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1331/2645] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:51.165509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661def29-7413-48ca-934f-667c57d2405a None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:51.166049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661def29-7413-48ca-934f-667c57d2405a None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:51.166049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661def29-7413-48ca-934f-667c57d2405a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:51.198752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-661def29-7413-48ca-934f-667c57d2405a None None] MFA Rules not processed for user `61dcabaa88464ba2aee72c163414d327`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:51.200322 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-661def29-7413-48ca-934f-667c57d2405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:51.211681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:51.276760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1315/2646] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:51.291663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:51.293398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:51.378490 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7efc943-2d5e-479f-945d-8e6e773e6ec5 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:51.388736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df6b286b-818e-4c10-8c85-b7284385f5ed service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:51.389556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b286b-818e-4c10-8c85-b7284385f5ed service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:51.390463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b286b-818e-4c10-8c85-b7284385f5ed service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:51.390463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df6b286b-818e-4c10-8c85-b7284385f5ed service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:51.403430 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b286b-818e-4c10-8c85-b7284385f5ed service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:51.405355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df6b286b-818e-4c10-8c85-b7284385f5ed service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:51.421335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1332/2647] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:51 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:53.622528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661def29-7413-48ca-934f-667c57d2405a None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:53.624579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-661def29-7413-48ca-934f-667c57d2405a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:53.646429 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661def29-7413-48ca-934f-667c57d2405a None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:53.653300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:53.654291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:53.654796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:53.655282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] PATH_INFO: `/v3/users/1cd38482beb94f7a94d9e33376424d67` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:53.656806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] RBAC: Authorizing `identity:delete_user(user_id=1cd38482beb94f7a94d9e33376424d67)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:53.662187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:53.918024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1cd38482beb94f7a94d9e33376424d67'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:53.919964 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1cd38482beb94f7a94d9e33376424d67'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:53.975779 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1cd38482beb94f7a94d9e33376424d67'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:53.994801 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1cd38482beb94f7a94d9e33376424d67'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.033424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1cd38482beb94f7a94d9e33376424d67'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.039806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1316/2648] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:53 2022] DELETE /identity/v3/users/1cd38482beb94f7a94d9e33376424d67 => generated 0 bytes in 418 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:54.461015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:54.461015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:54.461015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:54.471026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] MFA Rules not processed for user `61dcabaa88464ba2aee72c163414d327`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:54.472304 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:54.486530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:54.488137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:54.584215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1333/2649] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:54 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:54.595477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccd460f3-7913-4d1f-b5c3-e0bbd61eaf8c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:54.609190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:54.610085 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:54.610374 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:54.610657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] PATH_INFO: `/v3/projects/3acb29520eb94e5a978e31f0b2ac0b1e` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:54.611956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3acb29520eb94e5a978e31f0b2ac0b1e)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:54.613851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:54.725936 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:54.746411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3acb29520eb94e5a978e31f0b2ac0b1e', 'request_id': 'req-98736e2e-be5f-4c2f-987f-46d84241c7d2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.753214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3acb29520eb94e5a978e31f0b2ac0b1e', 'request_id': 'req-98736e2e-be5f-4c2f-987f-46d84241c7d2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.803799 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3acb29520eb94e5a978e31f0b2ac0b1e', 'request_id': 'req-98736e2e-be5f-4c2f-987f-46d84241c7d2'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.820711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] The token cache is being invalidate because project 3acb29520eb94e5a978e31f0b2ac0b1e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:54.821339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.822287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:54.824314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1317/2650] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:54 2022] DELETE /identity/v3/projects/3acb29520eb94e5a978e31f0b2ac0b1e => generated 0 bytes in 338 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:54.828856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:54.843358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2aea41b-ca39-4073-91ca-7261d4fdc59a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:54.848366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:54.922837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:54.923888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:54.924209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:54.924482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] PATH_INFO: `/v3/users/e5817d7f7f8640a88edb1136e93e3f43` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:54.925865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e5817d7f7f8640a88edb1136e93e3f43)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:54.927809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:55.169336 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e5817d7f7f8640a88edb1136e93e3f43'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.170158 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e5817d7f7f8640a88edb1136e93e3f43'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.266986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e5817d7f7f8640a88edb1136e93e3f43'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.274377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e5817d7f7f8640a88edb1136e93e3f43'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.286521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a861a257-260b-42b7-a615-3ee33e4f6a06 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e5817d7f7f8640a88edb1136e93e3f43'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.292377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1334/2651] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:54 2022] DELETE /identity/v3/users/e5817d7f7f8640a88edb1136e93e3f43 => generated 0 bytes in 464 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:55.566084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:55.567502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:55.583806 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98736e2e-be5f-4c2f-987f-46d84241c7d2 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:55.640841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:55.641717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:55.642003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:55.642279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] PATH_INFO: `/v3/projects/04e27a4290da4fe7947b95cda6467f39` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:55.643530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=04e27a4290da4fe7947b95cda6467f39)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:55.645493 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:55.815386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '04e27a4290da4fe7947b95cda6467f39', 'request_id': 'req-9ae85e25-bf95-450c-90a3-fdde77e059d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.820883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '04e27a4290da4fe7947b95cda6467f39', 'request_id': 'req-9ae85e25-bf95-450c-90a3-fdde77e059d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.911545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '04e27a4290da4fe7947b95cda6467f39', 'request_id': 'req-9ae85e25-bf95-450c-90a3-fdde77e059d3'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.924117 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] The token cache is being invalidate because project 04e27a4290da4fe7947b95cda6467f39 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:55.924409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.925140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9ae85e25-bf95-450c-90a3-fdde77e059d3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:55.926848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1318/2652] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:55 2022] DELETE /identity/v3/projects/04e27a4290da4fe7947b95cda6467f39 => generated 0 bytes in 361 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:03:55.931789 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:55.932299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:55.932399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:55.950729 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:55.951954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:56.012457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1335/2653] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:56.020577 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.021167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.021454 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.040069 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:56.041278 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:56.049775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1319/2654] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:56.054028 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.064389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ded1bf20-e6ab-48fa-bc11-e22e0a75993a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.069017 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.069621 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.069907 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.070184 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.070912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:03:56.071231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.072860 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.084774 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1336/2655] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:03:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:03:56.089058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.098751 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea5c9a5e-821c-4050-b42e-fa59a19c0a39 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.103480 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.104166 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.104448 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.104722 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.105945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.107850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.205002 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1320/2656] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:56 2022] POST /identity/v3/projects => generated 383 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:03:56.209178 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.220098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99d431dd-0e26-4297-b471-11c183980c4f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.229401 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:56.276722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.278130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.278722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.279384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.280384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.282425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.376482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1337/2657] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:56 2022] POST /identity/v3/users => generated 292 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:03:56.380364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.390953 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44576db7-b854-4f54-9100-84d081c48a25 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.396063 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.396739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.397022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.397304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.398041 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.399632 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.411563 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1321/2658] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:56.415591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.427666 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-859d4fed-5d82-4587-8bbc-ce21edf0caa6 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.434108 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f40e2b-8101-4052-ac62-fcf473a726c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.434874 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f40e2b-8101-4052-ac62-fcf473a726c7 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.435167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f40e2b-8101-4052-ac62-fcf473a726c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.435902 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f40e2b-8101-4052-ac62-fcf473a726c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.440925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f40e2b-8101-4052-ac62-fcf473a726c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.440925 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f40e2b-8101-4052-ac62-fcf473a726c7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.452584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1338/2659] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:56 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:56.453815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.463426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:DELETE=13,keystone:SELECT=84,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:56.468211 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e8794a-096b-49a4-a960-e3f4e30e0126 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.475726 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.476664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.477092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.477373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] PATH_INFO: `/v3/projects/cb5774cb52b04855aec4a31bd717780c/users/030def968e5a4a87862d66c391e95818/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.494800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] RBAC: Authorizing `identity:create_grant(project_id=cb5774cb52b04855aec4a31bd717780c, user_id=030def968e5a4a87862d66c391e95818, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.505059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.605359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1322/2660] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:56 2022] PUT /identity/v3/projects/cb5774cb52b04855aec4a31bd717780c/users/030def968e5a4a87862d66c391e95818/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:56.616969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.617200 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.617489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.643912 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] MFA Rules not processed for user `030def968e5a4a87862d66c391e95818`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:56.645125 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:56.712654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1339/2661] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:56 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:56.724408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.726107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:56.782674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-661fa6c7-9c74-4cb2-bf67-02e4982dde94 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.786321 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6455cfe8-6119-491c-9da0-a244107c8c47 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.786980 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455cfe8-6119-491c-9da0-a244107c8c47 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.787258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455cfe8-6119-491c-9da0-a244107c8c47 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.787527 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6455cfe8-6119-491c-9da0-a244107c8c47 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.796730 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455cfe8-6119-491c-9da0-a244107c8c47 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.798182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6455cfe8-6119-491c-9da0-a244107c8c47 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.807846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1323/2662] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:56 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:56.828126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:56.892690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1edbd7c7-8456-48e8-b88d-8ccb860ae73c None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:56.897190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:56.898062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:56.898346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:56.898623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] PATH_INFO: `/v3/users/030def968e5a4a87862d66c391e95818` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:56.899823 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] RBAC: Authorizing `identity:delete_user(user_id=030def968e5a4a87862d66c391e95818)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:56.901781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:56.924583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:57.063582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '030def968e5a4a87862d66c391e95818'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:57.064294 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '030def968e5a4a87862d66c391e95818'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:57.169812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '030def968e5a4a87862d66c391e95818'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:57.177008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '030def968e5a4a87862d66c391e95818'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:57.188986 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '030def968e5a4a87862d66c391e95818'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:57.195079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1340/2663] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:03:56 2022] DELETE /identity/v3/users/030def968e5a4a87862d66c391e95818 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:57.199597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:57.200075 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:57.200189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:57.217295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:57.218500 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:57.278485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1324/2664] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:57.287848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:57.289293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:57.346478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-444025a7-66e1-4c79-975f-c417c57c95da admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:57.350389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7285e0e4-0839-4097-8d8e-6981a2a886aa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:57.351055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285e0e4-0839-4097-8d8e-6981a2a886aa service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:57.351359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285e0e4-0839-4097-8d8e-6981a2a886aa service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:57.351638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7285e0e4-0839-4097-8d8e-6981a2a886aa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:57.360825 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285e0e4-0839-4097-8d8e-6981a2a886aa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:57.362259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7285e0e4-0839-4097-8d8e-6981a2a886aa service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:57.371611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1341/2665] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:03:57.866934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:57.868380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:57.883253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-234dc46d-1190-432d-81ec-1bb4bc4c2620 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:57.888407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:57.889312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:57.889593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:57.889883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] PATH_INFO: `/v3/projects/cb5774cb52b04855aec4a31bd717780c` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:57.891032 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] RBAC: Authorizing `identity:delete_project(project_id=cb5774cb52b04855aec4a31bd717780c)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:57.893017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.051238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'cb5774cb52b04855aec4a31bd717780c', 'request_id': 'req-7072cd86-67fb-47b2-b795-54140e022776'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:58.057699 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'cb5774cb52b04855aec4a31bd717780c', 'request_id': 'req-7072cd86-67fb-47b2-b795-54140e022776'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:58.152286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'cb5774cb52b04855aec4a31bd717780c', 'request_id': 'req-7072cd86-67fb-47b2-b795-54140e022776'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:58.164536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] The token cache is being invalidate because project cb5774cb52b04855aec4a31bd717780c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:03:58.164828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:58.165511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7072cd86-67fb-47b2-b795-54140e022776 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:03:58.167180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1325/2666] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:03:57 2022] DELETE /identity/v3/projects/cb5774cb52b04855aec4a31bd717780c => generated 0 bytes in 301 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:58.172929 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.173419 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.173517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.191742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:58.193287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:58.255259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1342/2667] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:58.263511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.264082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.264082 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.276811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=91,keystone:INSERT=2,keystone:DELETE=7 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:58.283400 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:58.284638 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:58.293927 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1326/2668] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:58.297948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:58.308014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81cc671d-a8c1-47d2-8c56-6bb6e781d5b2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:58.313267 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:58.313937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.314226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.314504 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.315366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:03:58.315552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:58.317196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.338389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1343/2669] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:03:58 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:03:58.339575 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:58.349603 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d61bb07-2f46-4845-ad5c-fcda0f4f8be5 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:58.356359 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:58.357348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.357896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.358371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.359662 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:58.361723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.482220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1327/2670] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:03:58 2022] POST /identity/v3/projects => generated 383 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:03:58.485644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:58.496694 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-610d30e6-f81e-481d-97cb-e5b24f7d3f81 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:58.531084 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:03:58.559356 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:58.559982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.560270 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.560549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.561279 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:58.563183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.685549 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1344/2671] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:03:58 2022] POST /identity/v3/users => generated 292 bytes in 200 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:03:58.693028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:58.700851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ac1a9e-4127-46b5-86cb-1bd5e9966e7b admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:58.705948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:58.706746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.707039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.707317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.708092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:58.709962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.726523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1328/2672] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:58.733584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:58.753155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64246ccf-907d-4d3f-9a0c-c501bdd1922e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:58.765042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f68a6d4c-51c7-4f32-8e90-f68c3ae4797b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:58.767379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a6d4c-51c7-4f32-8e90-f68c3ae4797b admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.767379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a6d4c-51c7-4f32-8e90-f68c3ae4797b admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.767379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f68a6d4c-51c7-4f32-8e90-f68c3ae4797b admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.767379 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a6d4c-51c7-4f32-8e90-f68c3ae4797b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:58.768937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f68a6d4c-51c7-4f32-8e90-f68c3ae4797b admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.780772 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1345/2673] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:03:58 2022] GET /identity/v3/roles => generated 1420 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:03:58.784714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:58.795194 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd01756-d01e-4636-ab66-8449bc2a8d89 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:58.800121 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:58.804244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.804244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.804244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] PATH_INFO: `/v3/projects/df5bb2aa3c1f48f79c8f0129194011dd/users/093620d84a384b95bbec11d62f9f1538/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.819255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] RBAC: Authorizing `identity:create_grant(project_id=df5bb2aa3c1f48f79c8f0129194011dd, user_id=093620d84a384b95bbec11d62f9f1538, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:58.822198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:58.924853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1329/2674] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:03:58 2022] PUT /identity/v3/projects/df5bb2aa3c1f48f79c8f0129194011dd/users/093620d84a384b95bbec11d62f9f1538/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:03:58.941786 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:58.942405 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:58.942760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:58.971070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] MFA Rules not processed for user `093620d84a384b95bbec11d62f9f1538`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:03:58.972341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:59.037076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1346/2675] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:03:58 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:03:59.049247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:03:59.052088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:03:59.123734 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cf84fb2-04a7-490b-900d-3ed9cc0edcc0 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:03:59.128183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:03:59.129134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:03:59.129247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:03:59.129623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:03:59.140916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:03:59.142487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:03:59.153736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1330/2676] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:03:59 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:07.674734 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:07.676338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:07.753831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3686ef02-30e7-4260-ab5d-64a76044076a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:07.757514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:07.758162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:07.758447 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:07.758721 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:07.760502 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:07.770683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:07.772182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:07.781164 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1347/2677] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:07.871276 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:07.873776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:07.887835 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030e324c-e617-4f52-b2c6-60a7ffb1b82a service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:07.891715 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:07.892407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:07.892690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:07.892999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:07.894899 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:07.977335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:07.978796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:07.991273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1331/2678] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:08.597975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:08.608182 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:08.670113 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:08.701521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac29fe7-791a-406d-9ce6-316331246797 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:08.706683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:08.707345 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:08.707635 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:08.707945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:08.709762 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:08.726223 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:08.727927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84426a1d-4ed7-46ee-92a8-a47a177ef77f service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:08.741797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1348/2679] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 145 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:09.137883 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:09.174056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:09.201584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f720df7-f03f-4892-b31a-b2bf787e69e7 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:09.206282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:09.207158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:09.207443 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:09.207723 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] PATH_INFO: `/v3/users/093620d84a384b95bbec11d62f9f1538` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:09.208950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] RBAC: Authorizing `identity:delete_user(user_id=093620d84a384b95bbec11d62f9f1538)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:09.210863 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:09.415496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '093620d84a384b95bbec11d62f9f1538'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:09.416516 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '093620d84a384b95bbec11d62f9f1538'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:09.489450 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '093620d84a384b95bbec11d62f9f1538'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:09.497593 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '093620d84a384b95bbec11d62f9f1538'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:09.509580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '093620d84a384b95bbec11d62f9f1538'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:09.516440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1332/2680] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:09 2022] DELETE /identity/v3/users/093620d84a384b95bbec11d62f9f1538 => generated 0 bytes in 379 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:09.521518 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca0978-2914-400f-b868-91266bdaa2a1 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:09.521855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca0978-2914-400f-b868-91266bdaa2a1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:09.524418 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baca0978-2914-400f-b868-91266bdaa2a1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:09.540918 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-baca0978-2914-400f-b868-91266bdaa2a1 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:09.547532 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-baca0978-2914-400f-b868-91266bdaa2a1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:09.601338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1349/2681] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:09.611192 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:09.612782 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:09.674536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7fc016d7-bf3f-4a1c-82d7-1c785ad3d14a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:09.678407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:09.679092 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:09.679381 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:09.679663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:09.693362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:09.695088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:09.706227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1333/2682] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:10.513520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c3b5a-e1a0-46be-950d-7beafdf32d77 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:10.514049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c3b5a-e1a0-46be-950d-7beafdf32d77 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:10.514149 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac8c3b5a-e1a0-46be-950d-7beafdf32d77 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:10.528060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:10.530488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:10.533623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ac8c3b5a-e1a0-46be-950d-7beafdf32d77 None None] MFA Rules not processed for user `f6e726651e4c49b794f5c86e1fe51d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:10.534934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ac8c3b5a-e1a0-46be-950d-7beafdf32d77 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:10.557145 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b12b094e-1dd5-4380-8db6-b1b4559ace04 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:10.563128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:10.564327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:10.564803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:10.565255 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] PATH_INFO: `/v3/projects/df5bb2aa3c1f48f79c8f0129194011dd` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:10.566640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] RBAC: Authorizing `identity:delete_project(project_id=df5bb2aa3c1f48f79c8f0129194011dd)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:10.568813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:10.612382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1350/2683] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:10.661747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:10.663122 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'df5bb2aa3c1f48f79c8f0129194011dd', 'request_id': 'req-80b34dbe-b2da-4fe0-9251-bcd112cf207c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:10.670183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'df5bb2aa3c1f48f79c8f0129194011dd', 'request_id': 'req-80b34dbe-b2da-4fe0-9251-bcd112cf207c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:10.740852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'df5bb2aa3c1f48f79c8f0129194011dd', 'request_id': 'req-80b34dbe-b2da-4fe0-9251-bcd112cf207c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:10.753068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] The token cache is being invalidate because project df5bb2aa3c1f48f79c8f0129194011dd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:10.753361 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:10.754023 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-80b34dbe-b2da-4fe0-9251-bcd112cf207c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:10.755853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1334/2684] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:10 2022] DELETE /identity/v3/projects/df5bb2aa3c1f48f79c8f0129194011dd => generated 0 bytes in 229 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:10.760777 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:10.762007 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:10.762299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:10.783250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:10.784818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:10.853205 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1351/2685] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:04:10.862844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:10.863183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:10.863392 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:10.884693 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:10.885989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:10.895988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1335/2686] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:10.900409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:10.911309 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c98057d-d1bd-4f72-88c8-e9bfa1196ef8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:10.917037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:10.917703 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:10.918566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:10.918566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:10.919235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:04:10.919593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:10.922006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:10.938876 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1352/2687] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:04:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jun 16 12:04:10.940066 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:10.950241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e34772b6-5f34-45a8-835e-5218ff6cce9c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:10.956207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:10.956884 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:10.957172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:10.957451 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:10.958205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:10.960182 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.079203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1336/2688] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:04:10 2022] POST /identity/v3/projects => generated 375 bytes in 137 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:04:11.080961 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.088286 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:11.092720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7022f95-65b6-4677-ac88-39bd37bb553e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.158030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.158728 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.161916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.161916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.161916 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.162216 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.245053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1353/2689] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:04:11 2022] POST /identity/v3/users => generated 288 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:11.246750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.257473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1c0b189-e9e2-40f5-9fc4-cd7487a470ea admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.263725 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.264414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.264761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.266254 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.267022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.268677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.285362 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1337/2690] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:04:11.286638 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.298026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa755b79-ba55-4c98-9e52-28578c43fb1c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.303039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0076ba59-e7b2-4fe5-99ca-c759d7bdddc7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.303939 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076ba59-e7b2-4fe5-99ca-c759d7bdddc7 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.304331 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076ba59-e7b2-4fe5-99ca-c759d7bdddc7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.304730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0076ba59-e7b2-4fe5-99ca-c759d7bdddc7 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.305592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076ba59-e7b2-4fe5-99ca-c759d7bdddc7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.307341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0076ba59-e7b2-4fe5-99ca-c759d7bdddc7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.325554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1354/2691] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:11.326843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.338129 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57fe3012-93a7-49b8-8e31-75b5dbb2c406 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.342864 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.344946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.345233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.346061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] PATH_INFO: `/v3/projects/c5fd0d847c664d3fb49d239b9723d51d/users/0befc21ffe4a44478c8a10af1b68e263/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.365004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] RBAC: Authorizing `identity:create_grant(project_id=c5fd0d847c664d3fb49d239b9723d51d, user_id=0befc21ffe4a44478c8a10af1b68e263, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.368320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.476287 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1338/2692] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:11 2022] PUT /identity/v3/projects/c5fd0d847c664d3fb49d239b9723d51d/users/0befc21ffe4a44478c8a10af1b68e263/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:11.485318 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.485618 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.485910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.518349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] MFA Rules not processed for user `0befc21ffe4a44478c8a10af1b68e263`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:11.519723 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:11.613632 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1355/2693] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:11 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:11.614907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.683444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdb30f5b-a184-4e41-ada6-4bce68f51ecd admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.692213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.692893 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.693175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.693521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.694149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.696159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.781006 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1339/2694] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:04:11 2022] POST /identity/v3/projects => generated 373 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:11.782212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.793153 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1fc800-cfaa-46e1-9848-d96d1589d001 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.867242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.867949 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.868238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.868582 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.869376 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.872163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.955910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1356/2695] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:04:11 2022] POST /identity/v3/users => generated 293 bytes in 172 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:04:11.958064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:11.970676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70c91dab-f2f0-427e-90d7-f3366951b740 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:11.976289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:11.976999 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:11.977335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:11.977618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:11.978350 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:11.980033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:11.995138 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1340/2696] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:11.996650 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:12.002535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:12.009480 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46162d2-7ed1-4ae8-be93-69d929b93f1f admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:12.014877 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:12.016034 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:12.016615 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:12.017048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] PATH_INFO: `/v3/projects/02e823b2540c4aa3af446b1263e155c6/users/4e33292e916a4fbabbf35ab936db735b/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:12.034451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e823b2540c4aa3af446b1263e155c6, user_id=4e33292e916a4fbabbf35ab936db735b, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:12.037301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:12.138597 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1357/2697] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:11 2022] PUT /identity/v3/projects/02e823b2540c4aa3af446b1263e155c6/users/4e33292e916a4fbabbf35ab936db735b/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 140 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:12.140170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:12.157530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1681afce-ed38-4cec-bce8-0be1b5c1f26d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:12.235737 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=93,keystone:DELETE=3,keystone:INSERT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:12.253521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41748bee-8a34-485f-aef7-e9ccebb6accf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:12.254401 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748bee-8a34-485f-aef7-e9ccebb6accf admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:12.254832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748bee-8a34-485f-aef7-e9ccebb6accf admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:12.255214 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41748bee-8a34-485f-aef7-e9ccebb6accf admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:12.256084 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41748bee-8a34-485f-aef7-e9ccebb6accf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:12.257811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41748bee-8a34-485f-aef7-e9ccebb6accf admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:12.276369 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1341/2698] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:12 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:12.281313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:12.291899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7465922e-cd2e-4d8c-9ee4-3524ad253345 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:12.301813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:12.303021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:12.303308 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:12.303592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] PATH_INFO: `/v3/projects/02e823b2540c4aa3af446b1263e155c6/users/4e33292e916a4fbabbf35ab936db735b/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:12.305799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] RBAC: Authorizing `identity:create_grant(project_id=02e823b2540c4aa3af446b1263e155c6, user_id=4e33292e916a4fbabbf35ab936db735b, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:12.308690 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:12.331116 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1358/2699] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:12 2022] PUT /identity/v3/projects/02e823b2540c4aa3af446b1263e155c6/users/4e33292e916a4fbabbf35ab936db735b/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:12.332690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:12.333059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:12.333416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:12.368444 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] MFA Rules not processed for user `4e33292e916a4fbabbf35ab936db735b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:12.369988 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:12.465421 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1342/2700] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:12 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 131 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:12.473587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:12.475586 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:12.539997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81034da0-0354-4cb4-b0c6-7bef50386144 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:12.557807 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-612c121e-6610-41c8-9d19-2ab2a78a4ca4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:12.558590 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c121e-6610-41c8-9d19-2ab2a78a4ca4 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:12.558940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c121e-6610-41c8-9d19-2ab2a78a4ca4 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:12.559334 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-612c121e-6610-41c8-9d19-2ab2a78a4ca4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:12.628254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c121e-6610-41c8-9d19-2ab2a78a4ca4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:12.629922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-612c121e-6610-41c8-9d19-2ab2a78a4ca4 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:12.640074 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1359/2701] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:12 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:15.731320 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:15.798050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6149e8a0-7390-4551-9a63-8583548607c3 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:15.802827 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:15.803752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:15.804207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:15.804492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] PATH_INFO: `/v3/users/0befc21ffe4a44478c8a10af1b68e263` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:15.805773 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0befc21ffe4a44478c8a10af1b68e263)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:15.807758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:16.015250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0befc21ffe4a44478c8a10af1b68e263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.017663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0befc21ffe4a44478c8a10af1b68e263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.083110 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0befc21ffe4a44478c8a10af1b68e263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.092051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0befc21ffe4a44478c8a10af1b68e263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.099348 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:16.106172 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0befc21ffe4a44478c8a10af1b68e263'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.112195 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1343/2702] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:15 2022] DELETE /identity/v3/users/0befc21ffe4a44478c8a10af1b68e263 => generated 0 bytes in 382 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:16.116785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:16.117258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:16.118039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:16.141850 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:16.143070 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:16.211188 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1360/2703] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:16.219330 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:16.220875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:16.279479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb1481c8-a3cd-4cce-991c-d038945e2620 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:16.283161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:16.283822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:16.284140 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:16.284423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:16.293837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:16.295548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:16.306180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1344/2704] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:16.691927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:16.693346 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:16.706699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00de1494-ef5f-44dc-8b21-ed8c61c5c6e8 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:16.711888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:16.712815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:16.713224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:16.713617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] PATH_INFO: `/v3/projects/c5fd0d847c664d3fb49d239b9723d51d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:16.714840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] RBAC: Authorizing `identity:delete_project(project_id=c5fd0d847c664d3fb49d239b9723d51d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:16.716887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:16.737592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:16.856095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c5fd0d847c664d3fb49d239b9723d51d', 'request_id': 'req-a3e6450c-7118-4d2e-87bd-dd6c2010487a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.862234 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c5fd0d847c664d3fb49d239b9723d51d', 'request_id': 'req-a3e6450c-7118-4d2e-87bd-dd6c2010487a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.867717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c5fd0d847c664d3fb49d239b9723d51d', 'request_id': 'req-a3e6450c-7118-4d2e-87bd-dd6c2010487a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.940111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] The token cache is being invalidate because project c5fd0d847c664d3fb49d239b9723d51d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:16.940410 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.941037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:16.942594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1361/2705] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:16 2022] DELETE /identity/v3/projects/c5fd0d847c664d3fb49d239b9723d51d => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:16.946613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:16.956479 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bd7a7ac-313d-403b-b145-beeedf38356b service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:17.012249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:17.013087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:17.013372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:17.013648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] PATH_INFO: `/v3/users/4e33292e916a4fbabbf35ab936db735b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:17.014821 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] RBAC: Authorizing `identity:delete_user(user_id=4e33292e916a4fbabbf35ab936db735b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:17.016808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:17.188606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4e33292e916a4fbabbf35ab936db735b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.189262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4e33292e916a4fbabbf35ab936db735b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.266352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4e33292e916a4fbabbf35ab936db735b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.275049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4e33292e916a4fbabbf35ab936db735b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.289376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-8126d74e-eed6-44dd-94fb-3ca7052c5e4b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4e33292e916a4fbabbf35ab936db735b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.296047 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1345/2706] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:16 2022] DELETE /identity/v3/users/4e33292e916a4fbabbf35ab936db735b => generated 0 bytes in 350 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:17.656649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:17.661176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:17.672684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3e6450c-7118-4d2e-87bd-dd6c2010487a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:17.731978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:17.732847 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:17.733134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:17.733411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] PATH_INFO: `/v3/projects/02e823b2540c4aa3af446b1263e155c6` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:17.734712 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=02e823b2540c4aa3af446b1263e155c6)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:17.736601 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:17.847682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '02e823b2540c4aa3af446b1263e155c6', 'request_id': 'req-496bbd77-bf57-44b5-862a-e1ab390e50b0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.855512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '02e823b2540c4aa3af446b1263e155c6', 'request_id': 'req-496bbd77-bf57-44b5-862a-e1ab390e50b0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.861911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '02e823b2540c4aa3af446b1263e155c6', 'request_id': 'req-496bbd77-bf57-44b5-862a-e1ab390e50b0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.952041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] The token cache is being invalidate because project 02e823b2540c4aa3af446b1263e155c6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:17.952333 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.952974 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-496bbd77-bf57-44b5-862a-e1ab390e50b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:17.954718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1362/2707] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:17 2022] DELETE /identity/v3/projects/02e823b2540c4aa3af446b1263e155c6 => generated 0 bytes in 298 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:17.959581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:17.960087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:17.960279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:17.979094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:17.980915 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:17.985736 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=88,keystone:DELETE=11,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:18.040672 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1346/2708] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:18.050092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.050455 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.050744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.068861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:18.070047 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:18.078023 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1363/2709] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:18.082181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.092266 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3566b439-e834-4ac9-84a7-1a2d5f96ada0 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.096895 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.097551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.097842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.098124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.099152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:04:18.099460 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.101104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.113238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1347/2710] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:04:18 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:04:18.117260 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.126277 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-babe445e-14d1-4180-ac8e-610701ad453a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.130899 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.131544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.131852 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.132157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.132848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.134785 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.210594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1364/2711] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:04:18 2022] POST /identity/v3/projects => generated 383 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:04:18.215217 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.225410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef2b931e-ca18-4d03-8994-a91bf17aee1f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.282410 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.283080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.283365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.283668 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.284390 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.286301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.363297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1348/2712] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:04:18 2022] POST /identity/v3/users => generated 292 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:18.367420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.374095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:18.377842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61fe431-5644-45b3-9f4c-0bb69bdfc60c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.382616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.383312 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.383606 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.383940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.386795 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.386905 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.398698 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1365/2713] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:18.403028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.414167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc143c24-0f8a-478d-add5-8cf7159171c8 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.418606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c660b707-34dc-4eeb-85a1-9ca5fa90b359 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.419509 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b707-34dc-4eeb-85a1-9ca5fa90b359 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.419862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b707-34dc-4eeb-85a1-9ca5fa90b359 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.420161 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c660b707-34dc-4eeb-85a1-9ca5fa90b359 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.420923 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660b707-34dc-4eeb-85a1-9ca5fa90b359 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.422515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c660b707-34dc-4eeb-85a1-9ca5fa90b359 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.433181 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1349/2714] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:18.437000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.447111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2970c152-15ff-48fe-bfc7-62f3859a6fbf admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.452603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.453416 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.453815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.454162 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] PATH_INFO: `/v3/projects/9f64132c2ed44ea9889f1169d1dc1740/users/4827ead1eee84ee9bac459c98e1eb709/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.470093 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] RBAC: Authorizing `identity:create_grant(project_id=9f64132c2ed44ea9889f1169d1dc1740, user_id=4827ead1eee84ee9bac459c98e1eb709, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.473110 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.563988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1366/2715] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:18 2022] PUT /identity/v3/projects/9f64132c2ed44ea9889f1169d1dc1740/users/4827ead1eee84ee9bac459c98e1eb709/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:04:18.574683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.574974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.575258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.601837 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] MFA Rules not processed for user `4827ead1eee84ee9bac459c98e1eb709`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:18.603064 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:18.666097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1350/2716] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:18 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:18.670514 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.734435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d96ecede-a506-45a0-a590-5863f79c58cd admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.739102 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.739760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.740096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.740382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.741097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.742997 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.810114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1367/2717] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:04:18 2022] POST /identity/v3/projects => generated 383 bytes in 140 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:18.814135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.820892 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:18.825045 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae3508ea-5c1c-491e-9372-f6a2b60e530f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.881355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.882055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.882349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.882630 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.883332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.885332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.946469 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1351/2718] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:04:18 2022] POST /identity/v3/users => generated 292 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:18.951063 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.961227 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74476730-ed4a-4734-88ae-58b7f77a5b08 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.965799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.966451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:18.966739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:18.967015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:18.967743 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:18.969358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:18.980454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1368/2719] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:18.984355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:18.994055 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5eee9cd7-413a-48f2-a676-780b6d646cf6 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:18.999203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-949cd4e8-7529-408b-b8a0-0324f8d4e1cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:18.999855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cd4e8-7529-408b-b8a0-0324f8d4e1cf admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.000491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cd4e8-7529-408b-b8a0-0324f8d4e1cf admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.000491 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-949cd4e8-7529-408b-b8a0-0324f8d4e1cf admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.001142 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cd4e8-7529-408b-b8a0-0324f8d4e1cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.002717 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-949cd4e8-7529-408b-b8a0-0324f8d4e1cf admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.015282 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1352/2720] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:18 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:19.017133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.026977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-730e386a-4497-4891-b4bf-3f17a3123ac9 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.031665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.032584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.032867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.033140 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] PATH_INFO: `/v3/projects/62bb97dbe2eb47dbb0ea88618257dbb6/users/43dcdb928f5644cd889145c5ded336b2/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.048298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] RBAC: Authorizing `identity:create_grant(project_id=62bb97dbe2eb47dbb0ea88618257dbb6, user_id=43dcdb928f5644cd889145c5ded336b2, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.050951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.125035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1369/2721] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:19 2022] PUT /identity/v3/projects/62bb97dbe2eb47dbb0ea88618257dbb6/users/43dcdb928f5644cd889145c5ded336b2/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:19.136290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.136571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.136850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.164108 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] MFA Rules not processed for user `43dcdb928f5644cd889145c5ded336b2`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:19.165327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:19.229478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1353/2722] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:19.234482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.294313 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1934ac9c-e3c9-4872-8408-97d4284c350c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.298863 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.299552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.299837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.300143 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.300837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.302717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.422237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1370/2723] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:04:19 2022] POST /identity/v3/projects => generated 383 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:19.426226 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.436840 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8a16081-7352-42d5-91bc-8e3d814dc41c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.493353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.493950 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.494237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.494510 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.495203 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.497134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.589621 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1354/2724] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:04:19 2022] POST /identity/v3/users => generated 298 bytes in 161 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:19.590985 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.597834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:19.601927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f19f2a-06a5-4729-8e09-56f159f3d705 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.607489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.608220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.608616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.608904 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.609633 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.611238 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.625459 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1371/2725] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:19.626586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.631985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:19.637466 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e148aff-1c48-449c-b92d-5d75d537f973 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.642309 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.643224 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.643511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.643787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] PATH_INFO: `/v3/projects/ebb9b115067644e7a6490692f59532d5/users/434bb968ba7b43c0b4b64fb8bed59361/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.660205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb9b115067644e7a6490692f59532d5, user_id=434bb968ba7b43c0b4b64fb8bed59361, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.662846 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.746535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1355/2726] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:19 2022] PUT /identity/v3/projects/ebb9b115067644e7a6490692f59532d5/users/434bb968ba7b43c0b4b64fb8bed59361/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:04:19.750414 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.760365 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a27e30c2-c5a3-4e68-81e7-5ee2442c4aed admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.816428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f116ccd-9891-4c0f-bbda-7dcdff7b876a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.817089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116ccd-9891-4c0f-bbda-7dcdff7b876a admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.817377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116ccd-9891-4c0f-bbda-7dcdff7b876a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.817652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f116ccd-9891-4c0f-bbda-7dcdff7b876a admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.818370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f116ccd-9891-4c0f-bbda-7dcdff7b876a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.820202 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f116ccd-9891-4c0f-bbda-7dcdff7b876a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.830742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1372/2727] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:19 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:04:19.834830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.846495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e8738a8-2073-4d44-8634-9120284cded3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:19.851415 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:19.855242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.855242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.855242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] PATH_INFO: `/v3/projects/ebb9b115067644e7a6490692f59532d5/users/434bb968ba7b43c0b4b64fb8bed59361/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.855242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb9b115067644e7a6490692f59532d5, user_id=434bb968ba7b43c0b4b64fb8bed59361, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:19.857567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:19.867887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1356/2728] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:19 2022] PUT /identity/v3/projects/ebb9b115067644e7a6490692f59532d5/users/434bb968ba7b43c0b4b64fb8bed59361/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:19.877812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:19.878175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:19.878463 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:19.904757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] MFA Rules not processed for user `434bb968ba7b43c0b4b64fb8bed59361`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:19.905955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:19.976254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1373/2729] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:19 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:19.987414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:19.989312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:20.045354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fb90eae-28e6-42c8-8f5e-f999e15c418f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:20.049551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:20.050188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:20.050472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:20.050747 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:20.106259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:20.107738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:20.116652 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1357/2730] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:19 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:21.956273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:21.957924 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:21.970135 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d24438d-a45e-45c9-9b3f-153780603470 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:21.975583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:21.975583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:21.975583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:21.975583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:22.034914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:22.036392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:22.045352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1374/2731] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:21 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:23.358956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:23.367655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8e274a4-99dd-4569-b5a2-dd11ff604930 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:23.371358 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37efa7f2-2ddf-4bce-b540-ec56a6044154 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:23.372039 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efa7f2-2ddf-4bce-b540-ec56a6044154 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:23.372326 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efa7f2-2ddf-4bce-b540-ec56a6044154 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:23.372607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37efa7f2-2ddf-4bce-b540-ec56a6044154 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:23.381680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efa7f2-2ddf-4bce-b540-ec56a6044154 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:23.383210 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37efa7f2-2ddf-4bce-b540-ec56a6044154 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:23.393487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1358/2732] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:23 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 35 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:28.007085 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:28.021827 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:28.072836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-263522f3-b891-4c7b-8b01-5c4077e83a68 service cinder] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:28.078708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:28.079600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:28.079909 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:28.080197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] PATH_INFO: `/v3/users/4827ead1eee84ee9bac459c98e1eb709` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:28.081393 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] RBAC: Authorizing `identity:delete_user(user_id=4827ead1eee84ee9bac459c98e1eb709)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:28.083395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:28.252338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4827ead1eee84ee9bac459c98e1eb709'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:28.252991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4827ead1eee84ee9bac459c98e1eb709'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:28.300769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4827ead1eee84ee9bac459c98e1eb709'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:28.308259 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4827ead1eee84ee9bac459c98e1eb709'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:28.321891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4827ead1eee84ee9bac459c98e1eb709'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:28.328682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1375/2733] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:28 2022] DELETE /identity/v3/users/4827ead1eee84ee9bac459c98e1eb709 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:28.333298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:28.333808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:28.333930 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:28.351215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:28.352461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:28.428250 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1359/2734] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:28.438199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:28.439643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:28.496911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0c31a1-0050-4b4e-9800-a0d8ccd2225c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:28.500717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:28.501367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:28.501651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:28.501959 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:28.510988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:28.512477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:28.521714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1376/2735] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:28 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:28.890606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:28.892167 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:28.904501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cc1e088-0a83-4a95-b37d-8dec18fe4ab8 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:28.909640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:28.910536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:28.910831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:28.911110 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] PATH_INFO: `/v3/projects/9f64132c2ed44ea9889f1169d1dc1740` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:28.912355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] RBAC: Authorizing `identity:delete_project(project_id=9f64132c2ed44ea9889f1169d1dc1740)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:28.914274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:28.915990 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:29.002891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9f64132c2ed44ea9889f1169d1dc1740', 'request_id': 'req-e521c8d5-2b60-41ea-b151-8e8575c8fc80'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.008383 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9f64132c2ed44ea9889f1169d1dc1740', 'request_id': 'req-e521c8d5-2b60-41ea-b151-8e8575c8fc80'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.054550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9f64132c2ed44ea9889f1169d1dc1740', 'request_id': 'req-e521c8d5-2b60-41ea-b151-8e8575c8fc80'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.065870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] The token cache is being invalidate because project 9f64132c2ed44ea9889f1169d1dc1740 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:29.066870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.066870 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.068336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1360/2736] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:28 2022] DELETE /identity/v3/projects/9f64132c2ed44ea9889f1169d1dc1740 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:29.072370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:29.082551 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33606e6-cbd8-4fbd-b1d8-a0435c9ea21b service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:29.114714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:29.139435 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:29.140332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:29.140660 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:29.140938 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] PATH_INFO: `/v3/users/43dcdb928f5644cd889145c5ded336b2` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:29.142120 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=43dcdb928f5644cd889145c5ded336b2)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:29.144107 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:29.302251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '43dcdb928f5644cd889145c5ded336b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.302906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '43dcdb928f5644cd889145c5ded336b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.368888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '43dcdb928f5644cd889145c5ded336b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.376377 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '43dcdb928f5644cd889145c5ded336b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.388272 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '43dcdb928f5644cd889145c5ded336b2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.394242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1377/2737] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:29 2022] DELETE /identity/v3/users/43dcdb928f5644cd889145c5ded336b2 => generated 0 bytes in 322 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:29.770963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:29.772487 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:29.787613 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e521c8d5-2b60-41ea-b151-8e8575c8fc80 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:29.844678 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:29.845549 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:29.845834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:29.846113 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] PATH_INFO: `/v3/projects/62bb97dbe2eb47dbb0ea88618257dbb6` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:29.847294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] RBAC: Authorizing `identity:delete_project(project_id=62bb97dbe2eb47dbb0ea88618257dbb6)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:29.849476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:29.963306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '62bb97dbe2eb47dbb0ea88618257dbb6', 'request_id': 'req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:29.969580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '62bb97dbe2eb47dbb0ea88618257dbb6', 'request_id': 'req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.037237 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '62bb97dbe2eb47dbb0ea88618257dbb6', 'request_id': 'req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.048663 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] The token cache is being invalidate because project 62bb97dbe2eb47dbb0ea88618257dbb6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:30.048959 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.049640 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.051229 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1361/2738] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:29 2022] DELETE /identity/v3/projects/62bb97dbe2eb47dbb0ea88618257dbb6 => generated 0 bytes in 281 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:30.055225 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:30.056654 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:30.069792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f15f65d-9656-4585-a03f-5fd502e0b1a4 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:30.125310 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:30.126186 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:30.126474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:30.126755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] PATH_INFO: `/v3/users/434bb968ba7b43c0b4b64fb8bed59361` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:30.127978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] RBAC: Authorizing `identity:delete_user(user_id=434bb968ba7b43c0b4b64fb8bed59361)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:30.130531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:30.301927 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '434bb968ba7b43c0b4b64fb8bed59361'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.302684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '434bb968ba7b43c0b4b64fb8bed59361'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.379736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '434bb968ba7b43c0b4b64fb8bed59361'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.382212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=80,keystone:DELETE=18,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:30.387788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '434bb968ba7b43c0b4b64fb8bed59361'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.399589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-e96d335c-e2ab-464a-9c90-b160c79eaed6 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '434bb968ba7b43c0b4b64fb8bed59361'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.405477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1378/2739] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:30 2022] DELETE /identity/v3/users/434bb968ba7b43c0b4b64fb8bed59361 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:30.787242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:30.788739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:30.802858 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4f4b40d-45ce-4ab7-9497-af51e816e3bb admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:30.821014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:30.860702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:30.861690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:30.861976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:30.862253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] PATH_INFO: `/v3/projects/ebb9b115067644e7a6490692f59532d5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:30.863442 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb9b115067644e7a6490692f59532d5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:30.865452 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:30.986788 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb9b115067644e7a6490692f59532d5', 'request_id': 'req-822b2093-0ba1-4bd0-9d4a-16bc9185630c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:30.992482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb9b115067644e7a6490692f59532d5', 'request_id': 'req-822b2093-0ba1-4bd0-9d4a-16bc9185630c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:31.088711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb9b115067644e7a6490692f59532d5', 'request_id': 'req-822b2093-0ba1-4bd0-9d4a-16bc9185630c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:31.101024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] The token cache is being invalidate because project ebb9b115067644e7a6490692f59532d5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:31.101314 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:31.101939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-822b2093-0ba1-4bd0-9d4a-16bc9185630c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:31.103478 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1362/2740] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:30 2022] DELETE /identity/v3/projects/ebb9b115067644e7a6490692f59532d5 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:31.108517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.108809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.109092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.126970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:31.128247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:31.185594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1379/2741] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:31.193152 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.193447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.193733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.212049 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:31.213251 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:31.221574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1363/2742] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:31.225867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.235357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-301a8ad4-cfbb-4d49-9f98-01c2bddaaa1b None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.239893 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.240558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.240848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.241136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.241861 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:04:31.242174 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.243770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:31.255054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1380/2743] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:04:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:04:31.259233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.268800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-216bd81f-3297-42ef-b2ea-5aa0497d634f None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.273447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.274093 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.274382 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.274661 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.275347 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.277909 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:31.364341 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1364/2744] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:04:31 2022] POST /identity/v3/projects => generated 389 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:31.368456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.378796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-023190bb-4286-4f73-8b5f-3627d9051763 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.434935 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.435605 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.435930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.436220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.436917 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.438832 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:31.540803 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1381/2745] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:04:31 2022] POST /identity/v3/users => generated 295 bytes in 173 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:04:31.544802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.555128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6991eedc-df13-4e50-ae3d-e1650e72d573 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.560325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.560977 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.561260 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.561571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.562284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.563859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:31.576290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1365/2746] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:04:31.580258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.590046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-448b94c3-bb51-442e-aa1d-0ff5fe1ab714 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.594692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf63090d-9b95-47ee-a2ca-8339f71bb609 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.595337 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63090d-9b95-47ee-a2ca-8339f71bb609 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.595644 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63090d-9b95-47ee-a2ca-8339f71bb609 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.595957 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf63090d-9b95-47ee-a2ca-8339f71bb609 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.596681 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63090d-9b95-47ee-a2ca-8339f71bb609 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.598264 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf63090d-9b95-47ee-a2ca-8339f71bb609 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:31.608209 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1382/2747] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:04:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:04:31.612105 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.622542 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e5dcbd6-d725-45d8-8098-30a9cf323eca admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.627050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.631687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.631687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.631687 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] PATH_INFO: `/v3/projects/5faaa9660bb24584bb62c9844774b20d/users/8bf1380803a447eebb73e7a61bf6d583/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.643771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=5faaa9660bb24584bb62c9844774b20d, user_id=8bf1380803a447eebb73e7a61bf6d583, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.646432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:31.752713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1366/2748] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:04:31 2022] PUT /identity/v3/projects/5faaa9660bb24584bb62c9844774b20d/users/8bf1380803a447eebb73e7a61bf6d583/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 141 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:31.816043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.816341 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.816626 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.843204 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] MFA Rules not processed for user `8bf1380803a447eebb73e7a61bf6d583`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:31.844449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:31.874060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=95,keystone:DELETE=2,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:31.908245 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1383/2749] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:31 2022] POST /identity/v3/auth/tokens => generated 3256 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:31.919419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:31.921164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:31.971844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:31.977380 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03abcbd9-e9fe-4dad-ae81-a914971085c1 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:31.981758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58a930a1-d6ac-45b5-b897-802e1245725c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:31.982394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a930a1-d6ac-45b5-b897-802e1245725c service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:31.982676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a930a1-d6ac-45b5-b897-802e1245725c service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:31.982957 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58a930a1-d6ac-45b5-b897-802e1245725c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:31.991078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a930a1-d6ac-45b5-b897-802e1245725c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:31.992537 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58a930a1-d6ac-45b5-b897-802e1245725c service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:32.002094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1367/2750] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:31 2022] GET /identity/v3/auth/tokens => generated 3256 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:41.974617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=14 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:42.053675 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:04:52.441089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:52.502503 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69d37f98-4b10-4d04-9a1c-0147519528c0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:52.506960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:52.507821 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:52.508141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:52.508421 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] PATH_INFO: `/v3/users/8bf1380803a447eebb73e7a61bf6d583` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:52.509588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf1380803a447eebb73e7a61bf6d583)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:52.511517 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:52.661251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf1380803a447eebb73e7a61bf6d583'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:52.661903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf1380803a447eebb73e7a61bf6d583'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:52.728697 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf1380803a447eebb73e7a61bf6d583'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:52.736756 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf1380803a447eebb73e7a61bf6d583'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:52.749001 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf1380803a447eebb73e7a61bf6d583'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:52.754880 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1384/2751] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:04:52 2022] DELETE /identity/v3/users/8bf1380803a447eebb73e7a61bf6d583 => generated 0 bytes in 314 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:04:52.759333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:52.759800 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:52.759921 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:52.776551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:04:52.777754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:52.837683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1368/2752] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:04:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:04:52.849059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:52.850683 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:52.911068 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57568245-39ed-416a-aba1-0013ae496e6e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:52.914754 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deec84cf-299e-4279-b984-306d3d729ceb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:52.915434 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec84cf-299e-4279-b984-306d3d729ceb service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:52.915722 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec84cf-299e-4279-b984-306d3d729ceb service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:52.916045 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deec84cf-299e-4279-b984-306d3d729ceb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:52.925461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec84cf-299e-4279-b984-306d3d729ceb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:52.926881 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deec84cf-299e-4279-b984-306d3d729ceb service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:52.936933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1385/2753] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:04:53.324183 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:04:53.326294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:04:53.339124 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30ad666c-df32-4919-807b-ae61ce5a3165 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:04:53.343764 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:04:53.344660 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:04:53.344945 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:04:53.345337 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] PATH_INFO: `/v3/projects/5faaa9660bb24584bb62c9844774b20d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:04:53.346528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] RBAC: Authorizing `identity:delete_project(project_id=5faaa9660bb24584bb62c9844774b20d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:04:53.348468 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:04:53.443308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '5faaa9660bb24584bb62c9844774b20d', 'request_id': 'req-d987424d-f725-4292-a1f6-5975db67b830'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:53.448803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '5faaa9660bb24584bb62c9844774b20d', 'request_id': 'req-d987424d-f725-4292-a1f6-5975db67b830'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:53.534007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '5faaa9660bb24584bb62c9844774b20d', 'request_id': 'req-d987424d-f725-4292-a1f6-5975db67b830'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:53.545305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] The token cache is being invalidate because project 5faaa9660bb24584bb62c9844774b20d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:04:53.545598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:53.546265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:04:53.547862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1369/2754] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:04:53 2022] DELETE /identity/v3/projects/5faaa9660bb24584bb62c9844774b20d => generated 0 bytes in 224 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:05:02.932043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=79,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:03.542078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=54,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:07.301117 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:07.301627 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:07.301736 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:07.319471 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] MFA Rules not processed for user `61dcabaa88464ba2aee72c163414d327`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:05:07.320695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:07.374250 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1386/2755] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:05:07 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:05:17.367335 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=25 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:43.852647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:43.853697 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:43.913205 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d987424d-f725-4292-a1f6-5975db67b830 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:43.916896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:43.917541 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:43.917824 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:43.918102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:43.919911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:43.979265 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:43.980852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:43.990507 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1370/2756] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:05:43 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:05:50.390920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:50.392842 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:50.404724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a76c29b-abe2-45c7-a213-ac7f5f815836 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:50.408920 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:50.409552 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:50.409834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:50.410111 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:50.411798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:50.475150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:50.476653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:50.486464 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1387/2757] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:05:53.985803 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=48 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:57.355985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:57.357557 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:57.372850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314efd5a-9896-4a6e-9131-a0b10992d973 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:57.378003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:57.378875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:57.379162 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:57.379435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] PATH_INFO: `/v3/users/6f81df1b996f4e6a808c83f4eebe586f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:57.380651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6f81df1b996f4e6a808c83f4eebe586f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:57.382567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:57.552574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6f81df1b996f4e6a808c83f4eebe586f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:57.553259 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6f81df1b996f4e6a808c83f4eebe586f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:57.641591 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6f81df1b996f4e6a808c83f4eebe586f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:57.648807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6f81df1b996f4e6a808c83f4eebe586f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:57.661028 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6f81df1b996f4e6a808c83f4eebe586f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:57.666910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1371/2758] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:05:57 2022] DELETE /identity/v3/users/6f81df1b996f4e6a808c83f4eebe586f => generated 0 bytes in 312 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:05:57.895709 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:57.906446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70ff1d1f-73a7-43b3-a3e2-b7082dd86761 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:57.969800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:57.971353 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:57.971642 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:57.971954 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] PATH_INFO: `/v3/projects/fab61ecc47674e6e961f67c0b412fda6` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:57.973163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] RBAC: Authorizing `identity:delete_project(project_id=fab61ecc47674e6e961f67c0b412fda6)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:57.975359 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:58.038315 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:58.039702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:58.055397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fab61ecc47674e6e961f67c0b412fda6', 'request_id': 'req-9b6759f7-76bf-4b80-837c-26590b364ece'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.061855 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fab61ecc47674e6e961f67c0b412fda6', 'request_id': 'req-9b6759f7-76bf-4b80-837c-26590b364ece'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.067449 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fab61ecc47674e6e961f67c0b412fda6', 'request_id': 'req-9b6759f7-76bf-4b80-837c-26590b364ece'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.098206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25c529e-9c06-4218-9153-4325f26fde95 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:58.109617 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] The token cache is being invalidate because project fab61ecc47674e6e961f67c0b412fda6 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:05:58.109915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.110565 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.112172 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1388/2759] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:05:57 2022] DELETE /identity/v3/projects/fab61ecc47674e6e961f67c0b412fda6 => generated 0 bytes in 217 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:05:58.116936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:58.118295 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:58.150533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:58.151185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:58.151589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:58.151766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:58.152099 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:58.153611 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:58.190142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b6759f7-76bf-4b80-837c-26590b364ece admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:58.196130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:58.197038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:58.197324 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:58.197602 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] PATH_INFO: `/v3/users/3d886cbef1ab4bd1a693df0daff30ad0` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:58.198602 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:58.198940 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3d886cbef1ab4bd1a693df0daff30ad0)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:58.201106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:58.227719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:58.229241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:58.241606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1372/2760] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:05:58 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 204 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jun 16 12:05:58.352793 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3d886cbef1ab4bd1a693df0daff30ad0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.353392 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3d886cbef1ab4bd1a693df0daff30ad0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.424584 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3d886cbef1ab4bd1a693df0daff30ad0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.431564 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3d886cbef1ab4bd1a693df0daff30ad0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.443849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3d886cbef1ab4bd1a693df0daff30ad0'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.450054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1389/2761] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:05:58 2022] DELETE /identity/v3/users/3d886cbef1ab4bd1a693df0daff30ad0 => generated 0 bytes in 333 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:05:58.723371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:58.736828 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16e6a9a3-bf66-4d22-8be6-a7f22e79f856 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:58.793887 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:58.794749 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:58.795034 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:58.795312 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] PATH_INFO: `/v3/projects/59052ab8010546d2adafb00ae8fdbd6d` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:58.796492 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] RBAC: Authorizing `identity:delete_project(project_id=59052ab8010546d2adafb00ae8fdbd6d)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:58.798916 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:58.903740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '59052ab8010546d2adafb00ae8fdbd6d', 'request_id': 'req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.909680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '59052ab8010546d2adafb00ae8fdbd6d', 'request_id': 'req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.956432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '59052ab8010546d2adafb00ae8fdbd6d', 'request_id': 'req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.967792 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] The token cache is being invalidate because project 59052ab8010546d2adafb00ae8fdbd6d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:05:58.968114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.968776 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-2c06defa-d3d9-4f8f-a912-cc11a87ae1ef admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:05:58.970853 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1373/2762] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:05:58 2022] DELETE /identity/v3/projects/59052ab8010546d2adafb00ae8fdbd6d => generated 0 bytes in 248 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:05:59.359670 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:05:59.361196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:59.418966 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37f2d818-f207-4174-aa92-1537c490b018 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:05:59.467840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:05:59.468524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:05:59.468814 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:05:59.469095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:05:59.471637 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:05:59.485060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:05:59.543014 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:05:59.544556 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8942dea9-1778-4d20-a30b-c37b716565ec service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:05:59.556511 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1390/2763] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:05:59 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 197 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:06:04.048447 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:06:04.049001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:06:04.049001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:06:04.068804 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] MFA Rules not processed for user `f6e726651e4c49b794f5c86e1fe51d9c`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:06:04.070128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:06:04.129561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1374/2764] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:06:04 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:06:09.619039 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=27 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:06:10.929096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:06:10.929719 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:06:10.929958 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:06:10.950984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] MFA Rules not processed for user `61dcabaa88464ba2aee72c163414d327`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:06:10.952255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:06:11.008790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1391/2765] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:06:10 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:06:13.166366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:06:13.167889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:06:13.179551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:06:13.184732 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f2a509d-50d6-4b53-982b-749d7118b208 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:06:13.190339 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:06:13.190987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:06:13.191275 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:06:13.191583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:06:13.193376 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:06:13.203607 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:06:13.205225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:06:13.214061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1375/2766] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:06:13 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:06:21.002091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=25 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:06:23.315531 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=4 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:07:11.186658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:07:11.188159 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:11.200771 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9ef89b-3865-466f-a5d4-d0c9b5eb4e56 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:07:11.204395 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:07:11.205066 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:11.208362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:11.208362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:11.208362 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:11.221444 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:07:11.222951 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e73955c6-2753-4086-b314-031805f8d045 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:07:11.232088 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1392/2767] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:07:11 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:07:15.451802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:07:15.464781 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31ad57af-10d4-4451-b9c6-6ad7a25ab13a service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:07:15.470319 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:07:15.472558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:15.472558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:15.472558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:15.473934 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:15.487336 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:07:15.488889 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:07:15.499831 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1376/2768] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:07:15 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:07:19.252693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:19.253181 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:19.254513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:19.270647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] MFA Rules not processed for user `61dcabaa88464ba2aee72c163414d327`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:07:19.271889 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:19.278862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1393/2769] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:07:19 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 27 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:07:19.573838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:07:19.575238 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:19.586963 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85a10a3a-8215-4132-a5fc-80eaf7627f27 service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:07:19.590551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:07:19.591187 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:19.591476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:19.591758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:19.593502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:19.603702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:07:19.605184 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:07:19.613353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1377/2770] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:07:19 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:07:20.002934 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:07:20.004840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:20.075031 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-757aabcd-3716-4d7c-aa58-445441dbad72 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:07:20.080055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:07:20.080708 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:20.080991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:20.081268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:20.083576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:20.094506 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:07:20.096096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:07:20.106233 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1394/2771] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:07:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:07:27.604585 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:07:27.618587 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a0037e3-dfcf-4920-945e-75323f00cbb5 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:07:27.624157 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:07:27.624802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:27.625135 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:27.625367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:27.627247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:27.639332 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:07:27.640987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:07:27.653078 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1378/2772] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:07:27 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:07:30.101815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=41 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:07:31.465478 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:07:31.466984 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:31.482408 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bda64484-ffce-47e4-8f35-faa06fa51f35 service placement] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:07:31.489241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:07:31.495755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:07:31.495755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:07:31.495755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:07:31.495755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:07:31.506241 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:07:31.507991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:07:31.520147 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1395/2773] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:07:31 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 55 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:07:37.647351 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=18 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:07:41.515669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=6 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:05.203325 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:05.212987 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e1badb-5525-40f2-9f42-dca819f4cf52 service nova] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:05.216711 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:05.217368 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:05.217649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:05.217924 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:05.219650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:05.282441 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:05.283903 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:05.293445 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1379/2774] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:08.686055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:08.699042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00eed7d3-08a9-46a7-8632-8026249c6d15 service nova] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:08.704154 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:08.704840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:08.705127 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:08.705411 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:08.707167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:08.719528 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:08.721126 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:08.732479 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1396/2775] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:08 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:12.034515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:12.036059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:12.048477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5cfb0f-2b35-4100-9cc6-4e6df423e8b5 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:12.053648 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:12.054529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:12.054817 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:12.055096 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] PATH_INFO: `/v3/users/f6e726651e4c49b794f5c86e1fe51d9c` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:12.056461 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f6e726651e4c49b794f5c86e1fe51d9c)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:12.058365 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:12.225212 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f6e726651e4c49b794f5c86e1fe51d9c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.225860 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f6e726651e4c49b794f5c86e1fe51d9c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.279770 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f6e726651e4c49b794f5c86e1fe51d9c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.287762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f6e726651e4c49b794f5c86e1fe51d9c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.299213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f6e726651e4c49b794f5c86e1fe51d9c'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.305158 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1380/2776] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:12 2022] DELETE /identity/v3/users/f6e726651e4c49b794f5c86e1fe51d9c => generated 0 bytes in 271 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:12.554114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:12.576896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41b37580-9f9c-44a2-bcb9-18bdad51326c service placement] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:12.636363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:12.637246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:12.637539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:12.637828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] PATH_INFO: `/v3/projects/4be7fa7276cb42d3a8b6dd0dd12cd784` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:12.639048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4be7fa7276cb42d3a8b6dd0dd12cd784)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:12.641055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:12.721009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4be7fa7276cb42d3a8b6dd0dd12cd784', 'request_id': 'req-180168ba-4163-42cb-8755-6d82ddfe0f1a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.727368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4be7fa7276cb42d3a8b6dd0dd12cd784', 'request_id': 'req-180168ba-4163-42cb-8755-6d82ddfe0f1a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.732915 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4be7fa7276cb42d3a8b6dd0dd12cd784', 'request_id': 'req-180168ba-4163-42cb-8755-6d82ddfe0f1a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.787947 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] The token cache is being invalidate because project 4be7fa7276cb42d3a8b6dd0dd12cd784 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:12.788253 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.788884 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:12.790508 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1397/2777] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:12 2022] DELETE /identity/v3/projects/4be7fa7276cb42d3a8b6dd0dd12cd784 => generated 0 bytes in 237 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:12.794692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:12.800021 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:12.810432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0b97e6c-56f6-4f36-ac3f-f0c88b7e5879 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:12.866657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:12.867515 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:12.867805 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:12.868120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] PATH_INFO: `/v3/users/7089d744edef40179e8bdf3b304fce5f` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:12.869342 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] RBAC: Authorizing `identity:delete_user(user_id=7089d744edef40179e8bdf3b304fce5f)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:12.871253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:12.876499 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:13.054633 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7089d744edef40179e8bdf3b304fce5f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.055594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7089d744edef40179e8bdf3b304fce5f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.119458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7089d744edef40179e8bdf3b304fce5f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.126614 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7089d744edef40179e8bdf3b304fce5f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.138760 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7089d744edef40179e8bdf3b304fce5f'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.144739 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1381/2778] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:12 2022] DELETE /identity/v3/users/7089d744edef40179e8bdf3b304fce5f => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:13.395541 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:13.397667 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:13.410682 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-180168ba-4163-42cb-8755-6d82ddfe0f1a admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:13.467473 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:13.468363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:13.468651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:13.468930 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] PATH_INFO: `/v3/projects/3be18f93e33a4a0cb91c958bf8bbcd4b` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:13.470128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] RBAC: Authorizing `identity:delete_project(project_id=3be18f93e33a4a0cb91c958bf8bbcd4b)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:13.472057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:13.492956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:13.570420 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3be18f93e33a4a0cb91c958bf8bbcd4b', 'request_id': 'req-2662cefe-deb3-440b-820c-027c162f200e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.576978 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3be18f93e33a4a0cb91c958bf8bbcd4b', 'request_id': 'req-2662cefe-deb3-440b-820c-027c162f200e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.582844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3be18f93e33a4a0cb91c958bf8bbcd4b', 'request_id': 'req-2662cefe-deb3-440b-820c-027c162f200e'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.644875 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] The token cache is being invalidate because project 3be18f93e33a4a0cb91c958bf8bbcd4b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:13.645173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.645799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:13.647849 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1398/2779] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:13 2022] DELETE /identity/v3/projects/3be18f93e33a4a0cb91c958bf8bbcd4b => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:23.141171 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=23,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:23.641725 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=16,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:29.858311 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:29.859918 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:29.938548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bee881b-ed36-43b2-8329-8c082319a63a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:29.994752 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:29.995550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:29.995834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:29.996144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] PATH_INFO: `/v3/users/61dcabaa88464ba2aee72c163414d327` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:29.997349 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] RBAC: Authorizing `identity:delete_user(user_id=61dcabaa88464ba2aee72c163414d327)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:29.999286 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:30.144464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '61dcabaa88464ba2aee72c163414d327'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.145120 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '61dcabaa88464ba2aee72c163414d327'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.216375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '61dcabaa88464ba2aee72c163414d327'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.223467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '61dcabaa88464ba2aee72c163414d327'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.235519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-ed9c0ec6-ca1f-49ae-9c10-ef2748e32b8e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '61dcabaa88464ba2aee72c163414d327'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.241458 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1382/2780] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:29 2022] DELETE /identity/v3/users/61dcabaa88464ba2aee72c163414d327 => generated 0 bytes in 384 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:30.475451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:30.477355 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:30.551348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2662cefe-deb3-440b-820c-027c162f200e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:30.611778 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:30.612684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:30.612969 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:30.613248 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] PATH_INFO: `/v3/projects/4e4082fe2ecb4756bdc57454c80d3b40` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:30.614431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e4082fe2ecb4756bdc57454c80d3b40)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:30.616911 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:30.731836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e4082fe2ecb4756bdc57454c80d3b40', 'request_id': 'req-a5afb94c-fb6d-4207-acec-b6920ac5f642'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.738009 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e4082fe2ecb4756bdc57454c80d3b40', 'request_id': 'req-a5afb94c-fb6d-4207-acec-b6920ac5f642'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.743950 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e4082fe2ecb4756bdc57454c80d3b40', 'request_id': 'req-a5afb94c-fb6d-4207-acec-b6920ac5f642'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.826820 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] The token cache is being invalidate because project 4e4082fe2ecb4756bdc57454c80d3b40 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:30.827081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.827647 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a5afb94c-fb6d-4207-acec-b6920ac5f642 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:30.829196 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1399/2781] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:30 2022] DELETE /identity/v3/projects/4e4082fe2ecb4756bdc57454c80d3b40 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:30.834257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:30.840068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:30.840068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:30.861600 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:30.862822 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:30.868582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:30.922573 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1383/2782] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:30.931384 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:30.934680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:30.934680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:30.951138 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:30.952397 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:30.960431 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1400/2783] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:30.964707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:30.974395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-811fff1a-b226-430d-9181-fcf92fab4e2d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:30.980702 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:30.981364 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:30.981649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:30.981925 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:30.982647 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:08:30.982956 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:30.984590 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:30.996650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1384/2784] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:08:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:08:31.000790 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.009994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e45521d-58e5-4938-98e0-6d07127db42e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.014739 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.015398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.015684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.016008 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.016744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.018652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:31.115175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1401/2785] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:30 2022] POST /identity/v3/projects => generated 373 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:08:31.119785 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.129825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fa3f951-9a3c-4159-8722-1468133f5138 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.185060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.185733 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.186017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.186294 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.186976 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.188928 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:31.262101 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1385/2786] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:31 2022] POST /identity/v3/users => generated 287 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:31.266261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.276755 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1afeca3b-1e8a-4b63-80f9-e481f9ce1a8d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.281349 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.282296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.282296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.282611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.283360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.284989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:31.286987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:31.297052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1402/2787] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:31.300802 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.311449 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60b1a96a-eff6-4b6a-ad3f-401e7b2988ba admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.316042 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e4692e8-d834-4ea3-8503-8b158ed04a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.316695 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4692e8-d834-4ea3-8503-8b158ed04a5c admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.316989 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4692e8-d834-4ea3-8503-8b158ed04a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.317256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e4692e8-d834-4ea3-8503-8b158ed04a5c admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.318094 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4692e8-d834-4ea3-8503-8b158ed04a5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.319582 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e4692e8-d834-4ea3-8503-8b158ed04a5c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:31.329384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1386/2788] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:31 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:31.333390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.343724 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60787b3a-aef0-4ec0-bf9d-cdd5bf4d6dba admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.348982 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.349853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.350141 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.351015 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] PATH_INFO: `/v3/projects/fa38fcf593f14dc1a22d63a4e9134f7e/users/463f8bb9792c4b359195f8e36692f767/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.365581 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] RBAC: Authorizing `identity:create_grant(project_id=fa38fcf593f14dc1a22d63a4e9134f7e, user_id=463f8bb9792c4b359195f8e36692f767, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.368192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:31.439477 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1403/2789] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:31 2022] PUT /identity/v3/projects/fa38fcf593f14dc1a22d63a4e9134f7e/users/463f8bb9792c4b359195f8e36692f767/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 106 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:31.450007 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.450300 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.450581 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.477763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] MFA Rules not processed for user `463f8bb9792c4b359195f8e36692f767`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:31.478962 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:31.542746 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1387/2790] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:31 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:31.553945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.555636 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:31.643087 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7994ded-65f5-4bae-a572-b45b6183ee5d admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.649006 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6d97f636-37a7-44ea-bb3e-efe2fc1f0769 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '54986964e91b4af7997e09d4682cc31e', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['admin', 'member', 'ResellerAdmin', 'reader', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.649648 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97f636-37a7-44ea-bb3e-efe2fc1f0769 service nova] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.649937 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97f636-37a7-44ea-bb3e-efe2fc1f0769 service nova] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.650213 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6d97f636-37a7-44ea-bb3e-efe2fc1f0769 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.657799 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97f636-37a7-44ea-bb3e-efe2fc1f0769 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.659357 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6d97f636-37a7-44ea-bb3e-efe2fc1f0769 service nova] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:31.668875 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1404/2791] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:31 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 115 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:31.884080 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:31.892856 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:31.947233 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b6d165-2448-43f8-9ad0-8c6326d6b689 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:31.951692 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:31.952584 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:31.952865 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:31.953160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] PATH_INFO: `/v3/users/463f8bb9792c4b359195f8e36692f767` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:31.954328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=463f8bb9792c4b359195f8e36692f767)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:31.956317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:32.155888 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '463f8bb9792c4b359195f8e36692f767'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:32.156580 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '463f8bb9792c4b359195f8e36692f767'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:32.245333 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '463f8bb9792c4b359195f8e36692f767'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:32.252728 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '463f8bb9792c4b359195f8e36692f767'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:32.265302 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '463f8bb9792c4b359195f8e36692f767'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:32.271284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1388/2792] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:31 2022] DELETE /identity/v3/users/463f8bb9792c4b359195f8e36692f767 => generated 0 bytes in 388 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:32.282707 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:32.283052 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:32.283321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:32.300587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:32.301798 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:32.359910 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1405/2793] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:32.369873 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:32.371305 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:32.448231 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253fb14a-dd91-47c4-a583-963aed8160b3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:32.451896 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be9c07f4-2b83-488f-9be6-fc1d74440243 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:32.452551 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c07f4-2b83-488f-9be6-fc1d74440243 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:32.452834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c07f4-2b83-488f-9be6-fc1d74440243 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:32.453109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be9c07f4-2b83-488f-9be6-fc1d74440243 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:32.461880 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c07f4-2b83-488f-9be6-fc1d74440243 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:32.463295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be9c07f4-2b83-488f-9be6-fc1d74440243 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:32.472545 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1389/2794] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:32.881190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:32.882651 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:32.896115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d69a0b3-69ae-40a5-bb24-6d6f05d8990e None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:32.901315 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:32.902299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:32.902600 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:32.902884 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] PATH_INFO: `/v3/projects/fa38fcf593f14dc1a22d63a4e9134f7e` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:32.904177 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fa38fcf593f14dc1a22d63a4e9134f7e)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:32.906089 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.021591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:33.028649 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fa38fcf593f14dc1a22d63a4e9134f7e', 'request_id': 'req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:33.034805 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fa38fcf593f14dc1a22d63a4e9134f7e', 'request_id': 'req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:33.040833 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fa38fcf593f14dc1a22d63a4e9134f7e', 'request_id': 'req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:33.119020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] The token cache is being invalidate because project fa38fcf593f14dc1a22d63a4e9134f7e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:33.119243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:33.119826 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-3acd5a43-4e7a-4fe1-94e5-0e30a93ea3a2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:33.121390 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1406/2795] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:32 2022] DELETE /identity/v3/projects/fa38fcf593f14dc1a22d63a4e9134f7e => generated 0 bytes in 241 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:33.126384 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.126855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.126855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.144786 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:33.145978 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:33.150571 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:33.205871 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1390/2796] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 80 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:33.214283 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.216285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.216285 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.233747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:33.235022 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:33.243555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1407/2797] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:08:33.248579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.259525 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0dad6769-259c-4cfb-8077-6f9beb10d503 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.264424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.265087 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.265408 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.265713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.266439 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:08:33.266753 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.268523 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.281529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1391/2798] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:08:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:08:33.285773 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.295352 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59240398-5ab2-4571-b472-bce23d2dcd1a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.300044 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.300699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.300989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.301271 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.301977 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.303894 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.366446 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1408/2799] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/projects => generated 365 bytes in 81 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:33.370705 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.380943 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d8bf743-b75d-47b7-a970-637421fed7ff admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.436218 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.436891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.437180 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.437462 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.438146 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.440097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.526221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1392/2800] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/users => generated 283 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:33.530257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.540672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e44f30ed-215b-4769-8a47-37de35e72833 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.546055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.546727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.547016 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.547298 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.548064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.549692 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.561378 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1409/2801] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:33.565310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.577657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38f07034-1fb3-4896-95de-d72cfb888f3f admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.581886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9712d2-c53b-4e41-a5fd-7f9578f458bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.584149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9712d2-c53b-4e41-a5fd-7f9578f458bd admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.584149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9712d2-c53b-4e41-a5fd-7f9578f458bd admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.584149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9712d2-c53b-4e41-a5fd-7f9578f458bd admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.584149 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9712d2-c53b-4e41-a5fd-7f9578f458bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.585422 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9712d2-c53b-4e41-a5fd-7f9578f458bd admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.595435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1393/2802] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:33 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:33.599554 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.610212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a84d1a-c032-46d9-95ad-60298cbc1a57 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.615092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.616037 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.616399 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.616686 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] PATH_INFO: `/v3/projects/4f53aa10bafb49a1b8e55069fe23f657/users/370770de2e5c43e48510d83a0108c9f6/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.632139 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f53aa10bafb49a1b8e55069fe23f657, user_id=370770de2e5c43e48510d83a0108c9f6, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.635483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.703543 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1410/2803] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:33 2022] PUT /identity/v3/projects/4f53aa10bafb49a1b8e55069fe23f657/users/370770de2e5c43e48510d83a0108c9f6/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:33.713912 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.714227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.714521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.740485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] MFA Rules not processed for user `370770de2e5c43e48510d83a0108c9f6`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:33.741686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:33.804159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1394/2804] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:33.809046 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.869639 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-007960da-1210-4650-a877-2244dc6882a5 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:33.874288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:33.874941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:33.875231 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:33.875513 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:33.876247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:33.878146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:33.955383 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1411/2805] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/projects => generated 369 bytes in 147 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:33.959416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:33.966090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:33.970558 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1760a82c-b655-4d1f-b36e-191b1a891c16 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.026424 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.027090 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.027373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.027649 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.028366 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.030263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.114159 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1395/2806] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:33 2022] POST /identity/v3/users => generated 291 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:34.118761 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:34.129000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193a28b-6fdf-4c5c-80e9-61ce7ea619cc admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.133684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.134339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.134695 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.134900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.135718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.137343 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.147623 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1412/2807] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:34.151501 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:34.161598 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d87d884-e805-4e51-a7f1-63c69f20e70c admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.166832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.167707 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.168024 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.168304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] PATH_INFO: `/v3/projects/01aedb2fd5584ad0bc2fa69f0030ed1d/users/87d5f8a9a6fa407684abc072afbd6de5/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.183308 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] RBAC: Authorizing `identity:create_grant(project_id=01aedb2fd5584ad0bc2fa69f0030ed1d, user_id=87d5f8a9a6fa407684abc072afbd6de5, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.185972 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.274060 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1396/2808] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:34 2022] PUT /identity/v3/projects/01aedb2fd5584ad0bc2fa69f0030ed1d/users/87d5f8a9a6fa407684abc072afbd6de5/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:08:34.281021 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:34.291669 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad8bb9fd-ae80-4668-8b8d-acedc5d01405 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.324693 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:34.346450 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcc29e4e-3872-4135-ad67-799e77473ee4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.347171 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc29e4e-3872-4135-ad67-799e77473ee4 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.347457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc29e4e-3872-4135-ad67-799e77473ee4 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.347732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcc29e4e-3872-4135-ad67-799e77473ee4 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.348475 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc29e4e-3872-4135-ad67-799e77473ee4 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.350098 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcc29e4e-3872-4135-ad67-799e77473ee4 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.360992 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1413/2809] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:34 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:08:34.364367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:34.374841 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f99e478-e142-4e64-b185-68f49520888a admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.379394 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295955-2a20-4064-9280-600431348374 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.380298 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295955-2a20-4064-9280-600431348374 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.380588 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295955-2a20-4064-9280-600431348374 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.380862 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf295955-2a20-4064-9280-600431348374 admin admin] PATH_INFO: `/v3/projects/01aedb2fd5584ad0bc2fa69f0030ed1d/users/87d5f8a9a6fa407684abc072afbd6de5/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.382818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295955-2a20-4064-9280-600431348374 admin admin] RBAC: Authorizing `identity:create_grant(project_id=01aedb2fd5584ad0bc2fa69f0030ed1d, user_id=87d5f8a9a6fa407684abc072afbd6de5, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.385467 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf295955-2a20-4064-9280-600431348374 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.395484 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1397/2810] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:34 2022] PUT /identity/v3/projects/01aedb2fd5584ad0bc2fa69f0030ed1d/users/87d5f8a9a6fa407684abc072afbd6de5/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:34.407442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.407735 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.408048 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.435809 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] MFA Rules not processed for user `87d5f8a9a6fa407684abc072afbd6de5`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:34.437054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:34.503288 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1414/2811] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:34 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:34.515088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295955-2a20-4064-9280-600431348374 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:34.516775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-cf295955-2a20-4064-9280-600431348374 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:34.573538 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf295955-2a20-4064-9280-600431348374 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.577130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.577771 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.578054 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.578329 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.587512 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.588960 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.598107 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1398/2812] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:34 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:34.692765 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:34.694151 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:34.706580 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c85cfb66-edbf-48e6-8046-97a4377fd156 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:34.710163 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-58eb2719-f3ba-4bf4-a5f4-607c6b5c587a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:34.713700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2719-f3ba-4bf4-a5f4-607c6b5c587a service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:34.713700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2719-f3ba-4bf4-a5f4-607c6b5c587a service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:34.713700 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58eb2719-f3ba-4bf4-a5f4-607c6b5c587a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:34.768092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb2719-f3ba-4bf4-a5f4-607c6b5c587a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:34.769524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-58eb2719-f3ba-4bf4-a5f4-607c6b5c587a service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:34.779035 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1415/2813] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:34 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:39.533834 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:39.594615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25ccac71-73ab-4997-a70b-2a2e2f01e913 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:39.599131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:39.600027 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:39.600310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:39.600583 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] PATH_INFO: `/v3/users/370770de2e5c43e48510d83a0108c9f6` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:39.602658 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=370770de2e5c43e48510d83a0108c9f6)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:39.604597 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:39.606050 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:39.773690 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '370770de2e5c43e48510d83a0108c9f6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:39.774378 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '370770de2e5c43e48510d83a0108c9f6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:39.861997 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '370770de2e5c43e48510d83a0108c9f6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:39.869221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '370770de2e5c43e48510d83a0108c9f6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:39.881189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '370770de2e5c43e48510d83a0108c9f6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:39.887131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1399/2814] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:39 2022] DELETE /identity/v3/users/370770de2e5c43e48510d83a0108c9f6 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:39.891613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:39.892146 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:39.892249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:39.909081 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:39.910268 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:39.958800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=100 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:39.968366 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1416/2815] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:39.978165 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:39.979606 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:40.035651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28565a41-81fd-4fd9-91db-4060cb6467a3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:40.039502 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:40.040177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:40.040464 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:40.040740 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:40.049521 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:40.050946 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:40.060328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1400/2816] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:39 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:40.486530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:40.488176 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:40.503474 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1eedb5a2-9835-4d73-bbb0-aec033bf3cf7 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:40.510194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:40.511224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:40.511652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:40.511836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] PATH_INFO: `/v3/projects/4f53aa10bafb49a1b8e55069fe23f657` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:40.512991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f53aa10bafb49a1b8e55069fe23f657)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:40.515321 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:40.604652 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f53aa10bafb49a1b8e55069fe23f657', 'request_id': 'req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:40.610744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f53aa10bafb49a1b8e55069fe23f657', 'request_id': 'req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:40.616237 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f53aa10bafb49a1b8e55069fe23f657', 'request_id': 'req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:40.690603 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] The token cache is being invalidate because project 4f53aa10bafb49a1b8e55069fe23f657 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:40.690926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:40.691546 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:40.693157 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1417/2817] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:40 2022] DELETE /identity/v3/projects/4f53aa10bafb49a1b8e55069fe23f657 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:40.697102 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:40.706942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70bfa563-1863-48b4-818b-93a83586f9cf service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:40.762411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:40.763241 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:40.763530 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:40.763813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] PATH_INFO: `/v3/users/87d5f8a9a6fa407684abc072afbd6de5` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:40.765030 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=87d5f8a9a6fa407684abc072afbd6de5)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:40.766939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:40.788757 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=85,keystone:DELETE=14,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:40.950637 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '87d5f8a9a6fa407684abc072afbd6de5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:40.951295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '87d5f8a9a6fa407684abc072afbd6de5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.021680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '87d5f8a9a6fa407684abc072afbd6de5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.029177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '87d5f8a9a6fa407684abc072afbd6de5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.041304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-9115d511-1250-4394-a25c-b09826008d94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '87d5f8a9a6fa407684abc072afbd6de5'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.047133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1401/2818] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:40 2022] DELETE /identity/v3/users/87d5f8a9a6fa407684abc072afbd6de5 => generated 0 bytes in 351 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:41.440398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:41.441856 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:41.455224 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7501868f-8740-4b2d-9a1a-4ae8028d1ecf admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:41.511897 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:41.512769 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:41.513059 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:41.513338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] PATH_INFO: `/v3/projects/01aedb2fd5584ad0bc2fa69f0030ed1d` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:41.514523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] RBAC: Authorizing `identity:delete_project(project_id=01aedb2fd5584ad0bc2fa69f0030ed1d)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:41.516484 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:41.617040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '01aedb2fd5584ad0bc2fa69f0030ed1d', 'request_id': 'req-32961579-613a-4250-a5c7-e7042532b282'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.623370 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '01aedb2fd5584ad0bc2fa69f0030ed1d', 'request_id': 'req-32961579-613a-4250-a5c7-e7042532b282'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.628845 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '01aedb2fd5584ad0bc2fa69f0030ed1d', 'request_id': 'req-32961579-613a-4250-a5c7-e7042532b282'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.690367 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] The token cache is being invalidate because project 01aedb2fd5584ad0bc2fa69f0030ed1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:41.690664 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.691293 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-32961579-613a-4250-a5c7-e7042532b282 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:41.692872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1418/2819] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:41 2022] DELETE /identity/v3/projects/01aedb2fd5584ad0bc2fa69f0030ed1d => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:41.698208 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:41.698664 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:41.698825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:41.745170 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:41.746375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:41.803750 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1402/2820] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:41.812076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:41.812371 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:41.812653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:41.830975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:41.832220 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:41.840523 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1419/2821] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:41.844440 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:41.855249 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c661a0d8-de10-4cd7-981d-7aeafaca4a24 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:41.859815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:41.860519 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:41.860809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:41.861088 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:41.861818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:08:41.862130 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:41.863729 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:41.875470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1403/2822] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:08:41 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:08:41.879249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:41.884815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:41.888534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e00a6c4f-912e-4127-89c9-d702df4bb834 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:41.893199 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:41.893844 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:41.894130 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:41.897183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:41.897183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:41.897183 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:41.979492 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1420/2823] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:41 2022] POST /identity/v3/projects => generated 399 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:41.983643 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:41.993984 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92096f63-6c86-4c5e-a0b7-56774ff5f909 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.052983 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.053653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.053942 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.054225 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.054917 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.056933 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.065204 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=91,keystone:DELETE=6,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:42.133826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1404/2824] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:41 2022] POST /identity/v3/users => generated 300 bytes in 149 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:42.136867 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.147468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-472beada-c6e4-4bbd-bf5d-a39d4ecbebfb admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.152858 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.153520 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.153810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.154095 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.154831 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.156465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.166465 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1421/2825] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:42.170338 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.180518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e86d7f-5936-41a2-9b33-d6e43e0ab332 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.185317 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b94ed4c-b3c4-4762-b4fe-3140b233490c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.185979 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94ed4c-b3c4-4762-b4fe-3140b233490c admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.186267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94ed4c-b3c4-4762-b4fe-3140b233490c admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.186548 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b94ed4c-b3c4-4762-b4fe-3140b233490c admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.187310 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94ed4c-b3c4-4762-b4fe-3140b233490c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.188951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b94ed4c-b3c4-4762-b4fe-3140b233490c admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.199579 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1405/2826] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:42.203596 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.214060 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-086255b6-9daa-4500-99e4-6fa497cba493 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.218714 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.219599 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.219914 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.220203 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] PATH_INFO: `/v3/projects/ebb13bda17904b1daa618270c1856067/users/33b867a1ae1c4583a1d3a136980b8f02/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.234840 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ebb13bda17904b1daa618270c1856067, user_id=33b867a1ae1c4583a1d3a136980b8f02, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.237521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.290451 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1422/2827] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:42 2022] PUT /identity/v3/projects/ebb13bda17904b1daa618270c1856067/users/33b867a1ae1c4583a1d3a136980b8f02/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 87 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:08:42.301708 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.302005 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.302289 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.328389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] MFA Rules not processed for user `33b867a1ae1c4583a1d3a136980b8f02`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:42.329599 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:42.393639 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1406/2828] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:42 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:42.398004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.458456 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b71cd246-f217-407c-afa3-de69fa19c319 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.463757 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.464442 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.464731 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.465071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.465760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.467680 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.532578 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1423/2829] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:42 2022] POST /identity/v3/projects => generated 399 bytes in 135 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:42.538017 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.549248 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c344bb81-aede-4098-9e5e-912ae426230d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.607761 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.608471 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.608763 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.609048 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.609758 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.611754 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.692244 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1407/2830] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:42 2022] POST /identity/v3/users => generated 306 bytes in 156 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:42.696249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.707020 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d94443dd-d603-4106-879e-09b4e2f267ab admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.711628 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.712322 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.712604 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.712888 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.713612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.715210 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.725796 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1424/2831] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:42.729859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.739656 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d5dbd00-973f-4de6-8b6b-eb798f630938 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.744809 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.745685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.745969 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.746253 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] PATH_INFO: `/v3/projects/a49dffed90e94b77a72b069ffa2193f7/users/6a0f93b36cf74e2093f43b404d6fec14/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.761354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49dffed90e94b77a72b069ffa2193f7, user_id=6a0f93b36cf74e2093f43b404d6fec14, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.764036 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.839004 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1408/2832] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:42 2022] PUT /identity/v3/projects/a49dffed90e94b77a72b069ffa2193f7/users/6a0f93b36cf74e2093f43b404d6fec14/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 110 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:08:42.842828 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.853075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37635a79-00cd-4d11-9b92-4a16648abf2c admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.910257 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d04951-e131-4d62-ab5a-fb906adf9006 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.910926 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d04951-e131-4d62-ab5a-fb906adf9006 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.911215 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d04951-e131-4d62-ab5a-fb906adf9006 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.911498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d04951-e131-4d62-ab5a-fb906adf9006 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.912256 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d04951-e131-4d62-ab5a-fb906adf9006 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.913887 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d04951-e131-4d62-ab5a-fb906adf9006 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.923891 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1425/2833] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:42 2022] GET /identity/v3/roles => generated 1420 bytes in 81 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:42.934173 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:42.944533 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954c7621-daea-4201-967a-0efe3a5b23aa admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:42.949134 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:42.950019 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.950290 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.950564 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] PATH_INFO: `/v3/projects/a49dffed90e94b77a72b069ffa2193f7/users/6a0f93b36cf74e2093f43b404d6fec14/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:42.952553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] RBAC: Authorizing `identity:create_grant(project_id=a49dffed90e94b77a72b069ffa2193f7, user_id=6a0f93b36cf74e2093f43b404d6fec14, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:42.955177 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:42.965220 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1409/2834] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:42 2022] PUT /identity/v3/projects/a49dffed90e94b77a72b069ffa2193f7/users/6a0f93b36cf74e2093f43b404d6fec14/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:42.974963 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545421e-6453-492c-aace-afac2883bec0 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:42.975282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545421e-6453-492c-aace-afac2883bec0 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:42.975567 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3545421e-6453-492c-aace-afac2883bec0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:43.002862 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-3545421e-6453-492c-aace-afac2883bec0 None None] MFA Rules not processed for user `6a0f93b36cf74e2093f43b404d6fec14`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:43.004145 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-3545421e-6453-492c-aace-afac2883bec0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:43.009092 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:43.071537 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1426/2835] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:42 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jun 16 12:08:43.081994 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:43.083650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:43.111263 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:43.142106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac5f7ee0-389d-45bf-b539-c0e71a3ba18e admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:43.146367 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:43.147008 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:43.147291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:43.147567 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:43.156848 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:43.158258 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:43.168046 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1410/2836] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:43 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:43.366922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545421e-6453-492c-aace-afac2883bec0 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:43.376738 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3545421e-6453-492c-aace-afac2883bec0 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:43.380394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9a1f409-b120-47fc-a192-8ddc58c74810 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:43.381030 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1f409-b120-47fc-a192-8ddc58c74810 service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:43.381326 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1f409-b120-47fc-a192-8ddc58c74810 service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:43.381589 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9a1f409-b120-47fc-a192-8ddc58c74810 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:43.437026 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1f409-b120-47fc-a192-8ddc58c74810 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:43.438448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9a1f409-b120-47fc-a192-8ddc58c74810 service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:43.447159 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1427/2837] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:43 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 81 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:48.298876 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:48.361189 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b45bac4-1652-48fb-879d-9bdce3a6392f service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:48.365738 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:48.366608 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:48.366897 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:48.367207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] PATH_INFO: `/v3/users/33b867a1ae1c4583a1d3a136980b8f02` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:48.368434 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=33b867a1ae1c4583a1d3a136980b8f02)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:48.370360 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:48.492891 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '33b867a1ae1c4583a1d3a136980b8f02'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:48.493574 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '33b867a1ae1c4583a1d3a136980b8f02'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:48.536562 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '33b867a1ae1c4583a1d3a136980b8f02'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:48.545012 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '33b867a1ae1c4583a1d3a136980b8f02'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:48.557419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '33b867a1ae1c4583a1d3a136980b8f02'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:48.563685 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1411/2838] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:48 2022] DELETE /identity/v3/users/33b867a1ae1c4583a1d3a136980b8f02 => generated 0 bytes in 265 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:48.568128 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:48.568566 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:48.568706 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:48.586054 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:48.587287 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:48.645594 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1428/2839] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:48.655163 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:48.656655 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:48.708528 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:48.713855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fa30fab-43c4-48ac-a332-f2abe1ffcf85 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:48.717594 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:48.718247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:48.718536 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:48.718844 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:48.728154 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:48.729626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:48.739003 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1412/2840] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:49.138539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:49.139994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:49.152812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eece5a5c-1dd9-4013-9669-0ec656207ea4 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:49.157521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:49.158388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:49.158674 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:49.158955 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] PATH_INFO: `/v3/projects/ebb13bda17904b1daa618270c1856067` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:49.160173 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ebb13bda17904b1daa618270c1856067)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:49.162079 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:49.265468 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:49.273730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ebb13bda17904b1daa618270c1856067', 'request_id': 'req-981da0ca-360b-4096-85db-f7641c6942c7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.279717 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ebb13bda17904b1daa618270c1856067', 'request_id': 'req-981da0ca-360b-4096-85db-f7641c6942c7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.285247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ebb13bda17904b1daa618270c1856067', 'request_id': 'req-981da0ca-360b-4096-85db-f7641c6942c7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.358261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] The token cache is being invalidate because project ebb13bda17904b1daa618270c1856067 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:49.358555 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.359228 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.360834 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1429/2841] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:49 2022] DELETE /identity/v3/projects/ebb13bda17904b1daa618270c1856067 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:49.364857 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:49.375014 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-146dc862-3b11-4432-b589-3699ba0d8fb1 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:49.429859 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:49.430653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:49.430939 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:49.431215 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] PATH_INFO: `/v3/users/6a0f93b36cf74e2093f43b404d6fec14` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:49.432898 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] RBAC: Authorizing `identity:delete_user(user_id=6a0f93b36cf74e2093f43b404d6fec14)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:49.434813 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:49.594653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '6a0f93b36cf74e2093f43b404d6fec14'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.595297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '6a0f93b36cf74e2093f43b404d6fec14'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.655589 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '6a0f93b36cf74e2093f43b404d6fec14'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.662691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '6a0f93b36cf74e2093f43b404d6fec14'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.675267 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-131b2475-aec5-49e6-a38d-7cf2ac2c8c83 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '6a0f93b36cf74e2093f43b404d6fec14'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:49.681518 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1413/2842] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:49 2022] DELETE /identity/v3/users/6a0f93b36cf74e2093f43b404d6fec14 => generated 0 bytes in 317 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jun 16 12:08:50.115055 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.116548 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:50.129612 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981da0ca-360b-4096-85db-f7641c6942c7 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.187792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.188699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.188991 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.189273 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] PATH_INFO: `/v3/projects/a49dffed90e94b77a72b069ffa2193f7` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.190457 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] RBAC: Authorizing `identity:delete_project(project_id=a49dffed90e94b77a72b069ffa2193f7)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.192409 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:50.296685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a49dffed90e94b77a72b069ffa2193f7', 'request_id': 'req-a71beb83-e641-4906-9525-f2a204206b3a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:50.302572 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a49dffed90e94b77a72b069ffa2193f7', 'request_id': 'req-a71beb83-e641-4906-9525-f2a204206b3a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:50.308043 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a49dffed90e94b77a72b069ffa2193f7', 'request_id': 'req-a71beb83-e641-4906-9525-f2a204206b3a'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:50.398672 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] The token cache is being invalidate because project a49dffed90e94b77a72b069ffa2193f7 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:50.398948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:50.399593 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-a71beb83-e641-4906-9525-f2a204206b3a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:50.401788 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1430/2843] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:50 2022] DELETE /identity/v3/projects/a49dffed90e94b77a72b069ffa2193f7 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:50.406604 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.407188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.407188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.424409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:50.425615 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:50.483926 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1414/2844] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:50.491752 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.496305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.496305 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.510812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:50.512053 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:50.519811 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1431/2845] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:50.523744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.534058 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-621896d6-24cc-48b8-b741-a15108952a3d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.538561 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.539213 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.539497 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.539775 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.540529 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:08:50.540838 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.542485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:50.544829 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:50.554198 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1415/2846] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:08:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:08:50.558558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.567836 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba49bcf-4b83-4e3a-b5ef-90e0332d9d92 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.573133 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.576657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.576657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.576657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.576657 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.577078 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:50.661242 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1432/2847] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:50 2022] POST /identity/v3/projects => generated 383 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:50.662404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.676073 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08424284-eecb-43a9-8ec9-2a83fc2fc12d admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.732546 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.733221 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.733513 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.733796 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.734481 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.736435 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:50.816670 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1416/2848] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:50 2022] POST /identity/v3/users => generated 292 bytes in 155 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:08:50.820673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.831890 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a38f5d1-2be0-4fdd-90c7-bad8349da49e admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.836587 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.837249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.837535 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.837815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.838611 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.840254 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:50.850425 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1433/2849] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:50.854386 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.864239 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d1a3b8b-ead7-43bb-90be-9a5e831377da admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.868842 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8035138-a9ca-47eb-a114-74bc42cba079 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.869986 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8035138-a9ca-47eb-a114-74bc42cba079 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.870274 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8035138-a9ca-47eb-a114-74bc42cba079 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.870553 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8035138-a9ca-47eb-a114-74bc42cba079 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.871279 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8035138-a9ca-47eb-a114-74bc42cba079 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.872910 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8035138-a9ca-47eb-a114-74bc42cba079 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:50.882830 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1417/2850] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:50 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jun 16 12:08:50.887558 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:50.897348 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40266629-c75b-4fb2-918c-8ad95c68b1c8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:50.902071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:50.902936 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:50.903219 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:50.903495 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] PATH_INFO: `/v3/projects/e08dd73c45214d75aa6110f39fa262ec/users/dd4fc62a607645359db7b5fc2612c97b/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:50.915112 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=86,keystone:DELETE=11,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:50.918166 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=e08dd73c45214d75aa6110f39fa262ec, user_id=dd4fc62a607645359db7b5fc2612c97b, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:50.920906 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.011460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1434/2851] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:50 2022] PUT /identity/v3/projects/e08dd73c45214d75aa6110f39fa262ec/users/dd4fc62a607645359db7b5fc2612c97b/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 124 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:51.022496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.022818 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.023104 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.050511 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] MFA Rules not processed for user `dd4fc62a607645359db7b5fc2612c97b`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:51.051759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:51.115235 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1418/2852] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:51 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:51.120222 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.179747 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21426a6f-365d-470d-ba21-5e2d108ef0b8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.185299 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.185960 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.186249 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.186531 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.187247 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.189194 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.263368 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1435/2853] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:51 2022] POST /identity/v3/projects => generated 381 bytes in 144 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:51.267419 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.278417 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f4fe8e7-579d-421c-8563-1e15ad1701b1 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.334470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.335144 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.335431 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.335710 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.336463 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.338375 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.346683 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:51.434676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1419/2854] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:51 2022] POST /identity/v3/users => generated 297 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:51.438524 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.448965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bae9aae-b414-4b43-8140-48a0f07cf847 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.453641 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.454281 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.454570 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.454854 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.455583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.457236 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.468853 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1436/2855] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:51.472855 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.483609 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8490134-26ea-4312-b191-533c9f5f61ab admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.488280 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.489197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.489488 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.489777 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] PATH_INFO: `/v3/projects/27f2eb25f5f8417ca7deee0f951783d2/users/230a43995c3441548d003bba7a29dcac/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.505190 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f2eb25f5f8417ca7deee0f951783d2, user_id=230a43995c3441548d003bba7a29dcac, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.507852 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.576207 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1420/2856] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:51 2022] PUT /identity/v3/projects/27f2eb25f5f8417ca7deee0f951783d2/users/230a43995c3441548d003bba7a29dcac/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 104 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:51.580091 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.590575 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7d384df-29ba-40cb-b8eb-c40d3384ef02 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.645530 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0c29908-b7f5-410d-9cc0-344b3cf7430c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.646197 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29908-b7f5-410d-9cc0-344b3cf7430c admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.646487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29908-b7f5-410d-9cc0-344b3cf7430c admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.646766 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0c29908-b7f5-410d-9cc0-344b3cf7430c admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.647487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c29908-b7f5-410d-9cc0-344b3cf7430c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.649115 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0c29908-b7f5-410d-9cc0-344b3cf7430c admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.659185 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1437/2857] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:51 2022] GET /identity/v3/roles => generated 1420 bytes in 80 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:51.663295 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.674128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8564714a-034a-470a-8c84-6d7d438ed1c3 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.678832 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.679759 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.680076 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.680355 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] PATH_INFO: `/v3/projects/27f2eb25f5f8417ca7deee0f951783d2/users/230a43995c3441548d003bba7a29dcac/roles/fda22db004a64c87b35510b321190ad4` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.682353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] RBAC: Authorizing `identity:create_grant(project_id=27f2eb25f5f8417ca7deee0f951783d2, user_id=230a43995c3441548d003bba7a29dcac, role_id=fda22db004a64c87b35510b321190ad4)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.685099 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.695352 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1421/2858] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:51 2022] PUT /identity/v3/projects/27f2eb25f5f8417ca7deee0f951783d2/users/230a43995c3441548d003bba7a29dcac/roles/fda22db004a64c87b35510b321190ad4 => generated 0 bytes in 32 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:51.705516 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.705941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.706235 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.733076 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] MFA Rules not processed for user `230a43995c3441548d003bba7a29dcac`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:51.734282 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:51.780829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:51.801240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1438/2859] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:51 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:51.815653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:51.817387 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:51.873402 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94a3568c-cc35-414b-b659-cf98392bfc10 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:51.877033 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:51.877681 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:51.877966 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:51.878243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:51.887774 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:51.889243 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:51.898534 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1422/2860] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:51 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:52.017970 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:52.028382 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20574dd9-21a5-4a6a-a7bd-44d65cc4bea1 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:52.031922 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47be7b59-21b8-4e13-b4df-c4354df1f3dd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:52.032583 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7b59-21b8-4e13-b4df-c4354df1f3dd service cinder] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:52.033944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7b59-21b8-4e13-b4df-c4354df1f3dd service cinder] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:52.033944 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47be7b59-21b8-4e13-b4df-c4354df1f3dd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:52.089588 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be7b59-21b8-4e13-b4df-c4354df1f3dd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:52.091032 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47be7b59-21b8-4e13-b4df-c4354df1f3dd service cinder] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:52.100105 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1439/2861] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:52 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:52.574304 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:52.635659 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a0f07c3-409c-49db-8dc3-e45bac72f631 service cinder] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:52.640168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:52.641022 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:52.641306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:52.641578 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] PATH_INFO: `/v3/users/dd4fc62a607645359db7b5fc2612c97b` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:52.642744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] RBAC: Authorizing `identity:delete_user(user_id=dd4fc62a607645359db7b5fc2612c97b)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:52.644666 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:52.829206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'dd4fc62a607645359db7b5fc2612c97b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:52.829819 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'dd4fc62a607645359db7b5fc2612c97b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:52.901894 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'dd4fc62a607645359db7b5fc2612c97b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:52.907353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:INSERT=4,keystone:SELECT=88,keystone:DELETE=8 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:52.909262 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'dd4fc62a607645359db7b5fc2612c97b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:52.921716 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'dd4fc62a607645359db7b5fc2612c97b'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:52.927485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1423/2862] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:52 2022] DELETE /identity/v3/users/dd4fc62a607645359db7b5fc2612c97b => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:52.932812 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:52.935482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:52.935482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:52.950136 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:52.951460 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:53.011658 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1440/2863] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:53.021623 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:53.023097 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:53.080261 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5656e6f-f29e-45cc-8c13-dcce161182bc admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:53.084331 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:53.084973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:53.085257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:53.088496 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:53.094913 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:53.096423 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:53.106851 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1424/2864] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:08:53.487025 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:53.491190 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:53.502137 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-157f8295-6f18-4e90-85e2-b2786232c29a None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:53.507086 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:53.508071 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:53.508428 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:53.508718 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] PATH_INFO: `/v3/projects/e08dd73c45214d75aa6110f39fa262ec` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:53.510454 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=e08dd73c45214d75aa6110f39fa262ec)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:53.512469 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:53.594941 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e08dd73c45214d75aa6110f39fa262ec', 'request_id': 'req-ca9acf45-42eb-419a-9329-dbb71d0177ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:53.601057 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e08dd73c45214d75aa6110f39fa262ec', 'request_id': 'req-ca9acf45-42eb-419a-9329-dbb71d0177ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:53.608500 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e08dd73c45214d75aa6110f39fa262ec', 'request_id': 'req-ca9acf45-42eb-419a-9329-dbb71d0177ec'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:53.667217 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] The token cache is being invalidate because project e08dd73c45214d75aa6110f39fa262ec was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:53.667512 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:53.668161 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:53.669760 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1441/2865] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:53 2022] DELETE /identity/v3/projects/e08dd73c45214d75aa6110f39fa262ec => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:53.673947 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:53.686109 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daeeb7be-0cbe-4935-846e-720c34698578 service neutron] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:53.742514 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:53.743389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:53.743676 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:53.744543 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] PATH_INFO: `/v3/users/230a43995c3441548d003bba7a29dcac` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:53.745735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] RBAC: Authorizing `identity:delete_user(user_id=230a43995c3441548d003bba7a29dcac)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:53.747650 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:53.966106 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '230a43995c3441548d003bba7a29dcac'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:53.966768 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '230a43995c3441548d003bba7a29dcac'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.074240 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '230a43995c3441548d003bba7a29dcac'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.081414 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '230a43995c3441548d003bba7a29dcac'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.093291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-61fa6d0b-b576-4320-8c49-bff4c6334276 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '230a43995c3441548d003bba7a29dcac'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.095477 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:54.099808 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1425/2866] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:08:53 2022] DELETE /identity/v3/users/230a43995c3441548d003bba7a29dcac => generated 0 bytes in 426 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:54.481547 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:54.483003 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:54.498239 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca9acf45-42eb-419a-9329-dbb71d0177ec admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:54.503613 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:54.555339 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:54.556246 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:54.556534 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:54.556815 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] PATH_INFO: `/v3/projects/27f2eb25f5f8417ca7deee0f951783d2` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:54.557989 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] RBAC: Authorizing `identity:delete_project(project_id=27f2eb25f5f8417ca7deee0f951783d2)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:54.559913 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:54.656732 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '27f2eb25f5f8417ca7deee0f951783d2', 'request_id': 'req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.662770 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '27f2eb25f5f8417ca7deee0f951783d2', 'request_id': 'req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.668994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '27f2eb25f5f8417ca7deee0f951783d2', 'request_id': 'req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.713948 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] The token cache is being invalidate because project 27f2eb25f5f8417ca7deee0f951783d2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87092) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:08:54.714243 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.714865 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-9ce1a798-4b75-4ecd-aadc-64cb368cf1bf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:08:54.716482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1442/2867] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:08:54 2022] DELETE /identity/v3/projects/27f2eb25f5f8417ca7deee0f951783d2 => generated 0 bytes in 235 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:54.721328 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:54.721780 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:54.721920 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:54.740409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:54.741605 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:54.799744 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1426/2868] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:54.807385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:54.807663 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:54.807987 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:54.825744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:54.826945 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:54.834782 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1443/2869] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:54.839206 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:54.848826 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae2f98c-cd84-47fa-8682-87489c8682c7 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:54.854199 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:54.854847 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:54.855133 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:54.855411 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] PATH_INFO: `/v3/domains` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:54.856164 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87091) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:08:54.856476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:54.858074 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:54.869327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1427/2870] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:08:54 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:08:54.873296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:54.882539 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddd688fb-dd8c-4378-aee8-ff962a9e45c2 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:54.887759 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:54.888441 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:54.888726 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:54.889004 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] PATH_INFO: `/v3/projects` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:54.889720 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:54.891620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:54.960900 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1444/2871] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:08:54 2022] POST /identity/v3/projects => generated 369 bytes in 88 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:54.965653 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:54.975476 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53a8906e-816d-4bc6-90f4-1b4602db6a85 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:55.030482 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:55.031148 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:55.031437 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:55.031719 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] PATH_INFO: `/v3/users` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:55.032430 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:55.034353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:55.119550 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1428/2872] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:08:54 2022] POST /identity/v3/users => generated 285 bytes in 154 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jun 16 12:08:55.123643 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:55.134121 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-935d50ac-a6af-4c20-a533-6cb84c9ea978 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:55.139311 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:55.139998 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:55.140290 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:55.140573 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:55.141388 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:55.142975 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:55.154097 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1445/2873] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:55.158051 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:55.167886 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c38bedb-2112-4077-ae3c-675c47739ee2 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:55.172472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7f47c12-2fd3-4e29-8aa5-c7d0daa9ca91 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:55.173242 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47c12-2fd3-4e29-8aa5-c7d0daa9ca91 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:55.173535 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47c12-2fd3-4e29-8aa5-c7d0daa9ca91 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:55.173815 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7f47c12-2fd3-4e29-8aa5-c7d0daa9ca91 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:55.174539 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47c12-2fd3-4e29-8aa5-c7d0daa9ca91 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:55.176188 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7f47c12-2fd3-4e29-8aa5-c7d0daa9ca91 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:55.185971 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1429/2874] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:08:55 2022] GET /identity/v3/roles => generated 1420 bytes in 28 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:08:55.189837 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:55.200180 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4cd522-017b-4477-b4eb-aca48dbcf653 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:55.204810 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:55.205685 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] REQUEST_METHOD: `PUT` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:55.205994 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:55.206261 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] PATH_INFO: `/v3/projects/dbd85113115c44abb5d9f37dc8d56699/users/d320404532a74f5f85749b8e63f3e7f7/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:55.221301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] RBAC: Authorizing `identity:create_grant(project_id=dbd85113115c44abb5d9f37dc8d56699, user_id=d320404532a74f5f85749b8e63f3e7f7, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:55.223972 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:55.285175 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1446/2875] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:08:55 2022] PUT /identity/v3/projects/dbd85113115c44abb5d9f37dc8d56699/users/d320404532a74f5f85749b8e63f3e7f7/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 96 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:08:55.345416 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faff3c-b586-4a94-b643-a4777869566d None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:55.345714 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faff3c-b586-4a94-b643-a4777869566d None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:55.346001 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15faff3c-b586-4a94-b643-a4777869566d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:55.373974 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-15faff3c-b586-4a94-b643-a4777869566d None None] MFA Rules not processed for user `d320404532a74f5f85749b8e63f3e7f7`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:08:55.375137 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-15faff3c-b586-4a94-b643-a4777869566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:55.428373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:55.437641 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1430/2876] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:08:55 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:08:55.450106 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:08:55.451829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:08:55.492042 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:08:55.529965 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8900a358-e5c4-44e1-9dfa-4890411240d1 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:08:55.533653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '35d68b3bee5445849c59da135df06fea', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:08:55.534301 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:08:55.534591 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:08:55.534872 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:08:55.543646 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:08:55.545104 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:08:55.553744 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1447/2877] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:08:55 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:09:05.484973 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=2 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:05.559350 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=19 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:14.421766 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faff3c-b586-4a94-b643-a4777869566d None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:14.423175 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-15faff3c-b586-4a94-b643-a4777869566d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:14.512191 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15faff3c-b586-4a94-b643-a4777869566d None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:14.517227 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:14.517878 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:14.518197 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:14.518485 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:14.520335 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:14.531651 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:14.533257 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-477c95ac-d003-4274-890a-4dff99ee88b7 service placement] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:14.543762 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1431/2878] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:09:14 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 122 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:09:24.538820 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=41 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:26.958448 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:27.020988 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f2fdca-5ce7-4b2a-9dea-e729cbf95c03 service glance] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:27.025608 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:27.026494 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:27.026763 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:27.027041 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] PATH_INFO: `/v3/users/d320404532a74f5f85749b8e63f3e7f7` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:27.028251 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d320404532a74f5f85749b8e63f3e7f7)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:27.030742 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:27.172629 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd320404532a74f5f85749b8e63f3e7f7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:27.173296 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd320404532a74f5f85749b8e63f3e7f7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:27.233550 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd320404532a74f5f85749b8e63f3e7f7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:27.240544 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd320404532a74f5f85749b8e63f3e7f7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:27.252427 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd320404532a74f5f85749b8e63f3e7f7'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:27.258363 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1448/2879] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:09:26 2022] DELETE /identity/v3/users/d320404532a74f5f85749b8e63f3e7f7 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:09:27.262833 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:27.263306 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:27.263409 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:27.281160 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:09:27.282371 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:27.338985 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1432/2880] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:09:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:09:27.349284 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:27.350684 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:27.410096 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f6dc179-2563-47f5-8a03-ec98c0398dc5 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:27.413727 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28d9a6a7-d1eb-4f74-a5e2-1db5da2e3eed service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:27.414364 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9a6a7-d1eb-4f74-a5e2-1db5da2e3eed service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:27.414653 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9a6a7-d1eb-4f74-a5e2-1db5da2e3eed service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:27.414933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28d9a6a7-d1eb-4f74-a5e2-1db5da2e3eed service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:27.423933 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9a6a7-d1eb-4f74-a5e2-1db5da2e3eed service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:27.425360 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28d9a6a7-d1eb-4f74-a5e2-1db5da2e3eed service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:27.434704 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1449/2881] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:09:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:09:27.812256 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:27.817068 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:27.826018 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bba1a0c7-c4fd-4a29-9e29-64292ea53788 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:27.831395 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:27.832301 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:27.832586 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:27.832861 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] PATH_INFO: `/v3/projects/dbd85113115c44abb5d9f37dc8d56699` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:27.834284 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=dbd85113115c44abb5d9f37dc8d56699)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:27.836327 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:27.955150 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'dbd85113115c44abb5d9f37dc8d56699', 'request_id': 'req-59e38b75-032d-4e06-8aa7-395f5e6c54b9'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:27.961131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'dbd85113115c44abb5d9f37dc8d56699', 'request_id': 'req-59e38b75-032d-4e06-8aa7-395f5e6c54b9'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:28.033186 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'dbd85113115c44abb5d9f37dc8d56699', 'request_id': 'req-59e38b75-032d-4e06-8aa7-395f5e6c54b9'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:28.044389 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] The token cache is being invalidate because project dbd85113115c44abb5d9f37dc8d56699 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:09:28.044677 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:28.045297 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-59e38b75-032d-4e06-8aa7-395f5e6c54b9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:28.046850 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1433/2882] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:09:27 2022] DELETE /identity/v3/projects/dbd85113115c44abb5d9f37dc8d56699 => generated 0 bytes in 236 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:09:28.051622 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.052142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.052258 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.078385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:09:28.078385 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:28.087841 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:28.128919 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1450/2883] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:09:28.136624 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.136992 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.137353 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.155825 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:09:28.157056 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:28.165052 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1434/2884] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:09:28.169049 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.179576 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f47facc-a4d2-44d8-8064-313cd7bd4151 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.184132 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.184781 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.185062 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.185338 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] PATH_INFO: `/v3/domains` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.186064 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=87092) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jun 16 12:09:28.186375 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.188616 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.200134 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1451/2885] 10.222.0.40 () {58 vars in 1152 bytes} [Thu Jun 16 12:09:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jun 16 12:09:28.204123 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.213291 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e781a52a-39ba-4bdd-9cdd-b121aef8e72c None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.217965 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.218620 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.218907 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.219185 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] PATH_INFO: `/v3/projects` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.219908 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.221807 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.285718 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1435/2886] 10.222.0.40 () {60 vars in 1152 bytes} [Thu Jun 16 12:09:28 2022] POST /identity/v3/projects => generated 373 bytes in 82 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:09:28.289625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.300276 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68931a80-fd9f-4639-9c53-58792f698d70 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.355691 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.356342 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.356625 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.356896 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] PATH_INFO: `/v3/users` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.357579 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.359999 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.421829 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1452/2887] 10.222.0.40 () {60 vars in 1143 bytes} [Thu Jun 16 12:09:28 2022] POST /identity/v3/users => generated 287 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jun 16 12:09:28.425991 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.436270 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d87f24-53ba-4ce1-9639-63a734d87d30 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.441472 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.442126 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.442407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.442713 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] PATH_INFO: `/v3/roles` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.443470 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.445079 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.455252 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1436/2888] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:09:28.459226 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.469398 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39e5f02d-46f6-45a0-87b4-718014500fa8 admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.474155 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b54b5f44-7495-4be9-8082-6dcf54b7b908 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.474818 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b5f44-7495-4be9-8082-6dcf54b7b908 admin admin] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.475109 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b5f44-7495-4be9-8082-6dcf54b7b908 admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.475394 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b54b5f44-7495-4be9-8082-6dcf54b7b908 admin admin] PATH_INFO: `/v3/roles` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.476152 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b5f44-7495-4be9-8082-6dcf54b7b908 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.478000 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b54b5f44-7495-4be9-8082-6dcf54b7b908 admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.488358 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1453/2889] 10.222.0.40 () {58 vars in 1121 bytes} [Thu Jun 16 12:09:28 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jun 16 12:09:28.492179 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.502875 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6e63e8a-25af-4090-b8c4-af72f4874e17 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.507691 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.508601 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] REQUEST_METHOD: `PUT` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.508890 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.509168 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] PATH_INFO: `/v3/projects/eecc4e71bb7d43b4b63b7c65a1121235/users/ac46f5295c704aa3b4117ad0f7ddc5dc/roles/c2d46bdeda48498b8c04ea76027be796` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.523426 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] RBAC: Authorizing `identity:create_grant(project_id=eecc4e71bb7d43b4b63b7c65a1121235, user_id=ac46f5295c704aa3b4117ad0f7ddc5dc, role_id=c2d46bdeda48498b8c04ea76027be796)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.526131 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.594061 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1437/2890] 10.222.0.40 () {60 vars in 1482 bytes} [Thu Jun 16 12:09:28 2022] PUT /identity/v3/projects/eecc4e71bb7d43b4b63b7c65a1121235/users/ac46f5295c704aa3b4117ad0f7ddc5dc/roles/c2d46bdeda48498b8c04ea76027be796 => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:09:28.605461 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] REQUEST_METHOD: `POST` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.605806 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.606038 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.632255 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.auth.core [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] MFA Rules not processed for user `ac46f5295c704aa3b4117ad0f7ddc5dc`. Rule list: `[]` (Enabled: `True`). {{(pid=87092) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:09:28.633487 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:28.695730 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1454/2891] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:09:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:09:28.706657 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.708372 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:28.731686 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:28.785354 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2806996f-1722-48b0-8434-25bfe0ca0a54 admin admin] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.789473 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64dd4d36-ec24-4b96-a07b-77c2301744c0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '72c24cb67566467dbdf65cfd44a59274', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.790114 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd4d36-ec24-4b96-a07b-77c2301744c0 service cinder] REQUEST_METHOD: `GET` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.790399 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd4d36-ec24-4b96-a07b-77c2301744c0 service cinder] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.790674 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64dd4d36-ec24-4b96-a07b-77c2301744c0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.799495 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd4d36-ec24-4b96-a07b-77c2301744c0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.800951 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64dd4d36-ec24-4b96-a07b-77c2301744c0 service cinder] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:28.810072 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1438/2892] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:09:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 104 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:09:28.834058 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:28.853212 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:28.895696 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0ed6a09-b88d-4225-925f-57b7c26bc2a6 None None] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:28.900167 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:28.901090 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] REQUEST_METHOD: `DELETE` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:28.901389 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:28.901665 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] PATH_INFO: `/v3/users/ac46f5295c704aa3b4117ad0f7ddc5dc` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:28.902848 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] RBAC: Authorizing `identity:delete_user(user_id=ac46f5295c704aa3b4117ad0f7ddc5dc)` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:28.904797 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:29.114903 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ac46f5295c704aa3b4117ad0f7ddc5dc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.115630 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ac46f5295c704aa3b4117ad0f7ddc5dc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.181679 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ac46f5295c704aa3b4117ad0f7ddc5dc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.188800 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ac46f5295c704aa3b4117ad0f7ddc5dc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.200813 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.notifications [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ac46f5295c704aa3b4117ad0f7ddc5dc'} {{(pid=87092) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.207240 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1455/2893] 10.222.0.40 () {58 vars in 1223 bytes} [Thu Jun 16 12:09:28 2022] DELETE /identity/v3/users/ac46f5295c704aa3b4117ad0f7ddc5dc => generated 0 bytes in 374 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:09:29.211787 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] REQUEST_METHOD: `POST` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:29.212128 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:29.212407 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:29.230646 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.auth.core [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] MFA Rules not processed for user `1361568ee63a477ea9c9be31d039c80d`. Rule list: `[]` (Enabled: `True`). {{(pid=87091) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jun 16 12:09:29.231843 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:29.289811 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1439/2894] 10.222.0.40 () {58 vars in 957 bytes} [Thu Jun 16 12:09:29 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jun 16 12:09:29.299412 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:29.300824 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:29.357482 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d147b1a4-6f59-4836-b41e-c4919f26d6db admin admin] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:29.361040 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '516b221cb1794430befadcde5f9fd77b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:29.361673 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:29.361956 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:29.362230 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:29.371150 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:29.372592 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:29.382332 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1456/2895] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:09:29 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:09:29.800735 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] Authenticating user token {{(pid=87091) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:09:29.802247 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.fernet_utils [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87091) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:09:29.815432 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a1a2265-10f3-4af2-9b08-331e3daef271 None None] Validating token access rules against request {{(pid=87091) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:09:29.820273 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1361568ee63a477ea9c9be31d039c80d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': 'c9c201c143164f7b8064c986ec08f397', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87091) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:09:29.821618 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:09:29.821911 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] SCRIPT_NAME: `/identity` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:09:29.822193 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] PATH_INFO: `/v3/projects/eecc4e71bb7d43b4b63b7c65a1121235` {{(pid=87091) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:09:29.823559 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=eecc4e71bb7d43b4b63b7c65a1121235)` {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:09:29.825503 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] RBAC: Authorization granted {{(pid=87091) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:09:29.924626 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'eecc4e71bb7d43b4b63b7c65a1121235', 'request_id': 'req-43fb2373-ef19-4a2b-a023-2f3475121bc6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.930059 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'eecc4e71bb7d43b4b63b7c65a1121235', 'request_id': 'req-43fb2373-ef19-4a2b-a023-2f3475121bc6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.987373 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'eecc4e71bb7d43b4b63b7c65a1121235', 'request_id': 'req-43fb2373-ef19-4a2b-a023-2f3475121bc6'} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:29.998756 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] The token cache is being invalidate because project eecc4e71bb7d43b4b63b7c65a1121235 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=87091) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jun 16 12:09:29.999680 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:30.000404 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG keystone.notifications [None req-43fb2373-ef19-4a2b-a023-2f3475121bc6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=87091) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jun 16 12:09:30.001948 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: [pid: 87091|app: 0|req: 1440/2896] 10.222.0.40 () {58 vars in 1232 bytes} [Thu Jun 16 12:09:29 2022] DELETE /identity/v3/projects/eecc4e71bb7d43b4b63b7c65a1121235 => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jun 16 12:09:39.378192 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=71,keystone:DELETE=10,keystone:INSERT=1 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:09:39.995667 dv-cinder-846146-2-iscsi devstack@keystone.service[87091]: DEBUG dbcounter [-] [87091] Writing DB stats keystone:SELECT=80,keystone:DELETE=7,keystone:INSERT=1 {{(pid=87091) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jun 16 12:14:17.381075 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] Authenticating user token {{(pid=87092) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jun 16 12:14:17.383792 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:14:17.451489 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9bd59c-0692-4443-8aad-5775df4820a2 service neutron] Validating token access rules against request {{(pid=87092) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jun 16 12:14:17.510498 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f482060b5ab3458ba78d94184d68f10d', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '4e5224f2718d4d228b36c920cab06d42', 'project_domain_id': 'default', 'roles': ['member', 'reader', 'service', 'admin'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=87092) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jun 16 12:14:17.511142 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] REQUEST_METHOD: `GET` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jun 16 12:14:17.511424 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] SCRIPT_NAME: `/identity` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jun 16 12:14:17.511699 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=87092) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jun 16 12:14:17.513483 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.fernet_utils [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=87092) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jun 16 12:14:17.524521 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jun 16 12:14:17.526114 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97a503d9-9e6b-47fa-978f-c3d7d68490b3 service placement] RBAC: Authorization granted {{(pid=87092) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jun 16 12:14:17.535767 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: [pid: 87092|app: 0|req: 1457/2897] 10.222.0.40 () {60 vars in 1365 bytes} [Thu Jun 16 12:14:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 155 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jun 16 12:14:27.531620 dv-cinder-846146-2-iscsi devstack@keystone.service[87092]: DEBUG dbcounter [-] [87092] Writing DB stats keystone:SELECT=58 {{(pid=87092) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}